Re: [Qemu-devel] [PATCH 0/7] Qtest driver framework

2018-07-11 Thread Emanuele
On 07/11/2018 04:00 PM, Stefan Hajnoczi wrote: On Mon, Jul 09, 2018 at 11:11:29AM +0200, Emanuele Giuseppe Esposito wrote: Basic framework steps are the following: - All nodes and edges are created in their respective machine/driver/test files - The framework starts QEMU and asks for a list of

Re: [Qemu-devel] [PATCH 2/7] tests/qgraph: pci-pc driver and interface nodes

2018-07-11 Thread Emanuele
On 07/11/2018 04:49 PM, Stefan Hajnoczi wrote: On Mon, Jul 09, 2018 at 11:11:31AM +0200, Emanuele Giuseppe Esposito wrote: -QPCIBus *qpci_init_pc(QTestState *qts, QGuestAllocator *alloc) +static void *qpci_get_driver(void *obj, const char *interface) { -QPCIBusPC *ret = g_new0(QPCIBusPC

Re: [Qemu-devel] [PATCH 7/7] tests/qgraph: sdhci test node

2018-07-11 Thread Emanuele
On 07/11/2018 05:15 PM, Stefan Hajnoczi wrote: On Mon, Jul 09, 2018 at 11:11:36AM +0200, Emanuele Giuseppe Esposito wrote: +/** + * Old sdhci_t structure: Do you intend to delete this comment before this series is merged? It seems like a TODO that doesn't need to be kept around.

Re: [Qemu-devel] [PATCH 3/7] tests/qgraph: sdhci driver and interface nodes

2018-07-11 Thread Emanuele
Hi Philippe, On 07/11/2018 10:13 PM, Philippe Mathieu-Daudé wrote: Hi Emanuele, On 07/09/2018 06:11 AM, Emanuele Giuseppe Esposito wrote: Add qgraph nodes for sdhci-pci and generic-sdhci (memory mapped) drivers. Both drivers implement (produce) the same interface sdhci, that provides the

Re: [Qemu-devel] [PATCH 1/7] tests: qgraph API for the qtest driver framework

2018-07-18 Thread Emanuele
On 07/18/2018 04:23 PM, Stefan Hajnoczi wrote: On Wed, Jul 11, 2018 at 04:58:41PM +0200, Paolo Bonzini wrote: On 11/07/2018 16:28, Stefan Hajnoczi wrote: + * + * QOSGraphObject also provides a destructor, used to deallocate the + * after the test has been executed. + */ +struct QOSGraphObjec

Re: [Qemu-devel] [PATCH 2/7] tests/qgraph: pci-pc driver and interface nodes

2018-07-18 Thread Emanuele
On 07/18/2018 04:29 PM, Stefan Hajnoczi wrote: On Wed, Jul 11, 2018 at 05:18:03PM +0200, Paolo Bonzini wrote: On 11/07/2018 16:49, Stefan Hajnoczi wrote: On Mon, Jul 09, 2018 at 11:11:31AM +0200, Emanuele Giuseppe Esposito wrote: -QPCIBus *qpci_init_pc(QTestState *qts, QGuestAllocator

Re: [Qemu-devel] [PATCH 2/7] tests/qgraph: pci-pc driver and interface nodes

2018-07-18 Thread Emanuele
On 07/18/2018 09:33 PM, Paolo Bonzini wrote: On 18/07/2018 20:29, Emanuele wrote: I had to put this patch here because it also introduces qpci_device_init, used by sdhci (patch 3). For the next version I plan to have a patch X where I rename all occurrences of qpci_init_pc in qpci_pc_new

Re: [Qemu-devel] [PATCH 1/7] tests: qgraph API for the qtest driver framework

2018-07-18 Thread Emanuele
On 07/18/2018 09:28 PM, Paolo Bonzini wrote: On 18/07/2018 16:23, Stefan Hajnoczi wrote: +struct QOSGraphObject { +/* for produces, returns void * */ +QOSGetDriver get_driver; Unused? +/* for contains, returns a QOSGraphObject * */ +QOSGetDevice get_device; Unused? What i

Re: [Qemu-devel] [PATCH v2 00/34] Qtest driver framework

2018-08-09 Thread Emanuele
On 08/09/2018 10:57 AM, Paolo Bonzini wrote: On 06/08/2018 16:33, Emanuele Giuseppe Esposito wrote: qgraph API for the qtest driver framework This series of patches introduce a different qtest driver organization, viewing machines, drivers and tests as node in a graph, each having one or

Re: [Qemu-devel] [PATCH v2 00/34] Qtest driver framework

2018-08-09 Thread Emanuele
On 08/09/2018 11:44 AM, Paolo Bonzini wrote: On 09/08/2018 11:20, Emanuele wrote: Why this? Shouldn't it be: if (g_strcmp0(old_path, path)) {     qtest_end(); /* handles global_qtest = NULL */     g_free(old_path); /* handles NULL */     old_path = path;     global_

Re: [Qemu-devel] [PATCH v2 03/34] tests/qgraph: pci-pc driver and interface nodes

2018-08-09 Thread Emanuele
dr->vendor_id); that should be in fact detected by this g_assert(). since this case is covered by qpci_device_available, I don't think there's the need to insert the assertion. Thank you, Emanuele

Re: [Qemu-devel] [PATCH v2 04/34] tests/qgraph: x86_64/pc machine node

2018-08-09 Thread Emanuele
", "memory", test_memory_allocator, NULL)? I don't know, is there a test for the memory allocator? I gave it that name just because the actual structure representing it is called QGuestAllocator. Thank you, Emanuele

Re: [Qemu-devel] [PATCH v2 03/34] tests/qgraph: pci-pc driver and interface nodes

2018-08-09 Thread Emanuele
On 08/09/2018 02:29 PM, Laurent Vivier wrote: On 09/08/2018 14:17, Emanuele wrote: +    return TRUE; +} + +QPCIDevice *qpci_device_find(QPCIBus *bus, int devfn) +{ +    QPCIDevice *dev; + +    dev = g_malloc0(sizeof(*dev)); + +    if (!qpci_device_set(dev, bus, devfn)) {   g_free

Re: [Qemu-devel] [PATCH v2 09/34] tests/qgraph: pci-spapr driver and interface nodes

2018-08-09 Thread Emanuele
On 08/09/2018 03:02 PM, Laurent Vivier wrote: On 06/08/2018 16:33, Emanuele Giuseppe Esposito wrote: +void qpci_init_spapr(QPCIBusSPAPR *ret, QTestState *qts, QGuestAllocator *alloc) +{ assert(qts); +/* tests cannot use spapr, needs to be fixed first */ +ret

Re: [Qemu-devel] [PATCH v2 05/34] tests/qgraph: sdhci driver and interface nodes

2018-08-09 Thread Emanuele
be *_destroy, no? Thank you, Emanuele

Re: [Qemu-devel] [PATCH v2 13/34] test/qgraph: virtio_start_device function

2018-08-09 Thread Emanuele
On 08/09/2018 03:39 PM, Laurent Vivier wrote: On 06/08/2018 16:33, Emanuele Giuseppe Esposito wrote: This function is intended to group all the qvirtio_* functions that start the qvirtio devices. Applied in all tests using this combination of functions. Signed-off-by: Emanuele Giuseppe

Re: [Qemu-devel] [PATCH v2 29/34] test/qgraph: virtio-blk test node

2018-08-10 Thread Emanuele
On 08/09/2018 04:16 PM, Laurent Vivier wrote: On 06/08/2018 16:34, Emanuele Giuseppe Esposito wrote: -static void mmio_basic(void) +static void basic_resize(void *obj, void *data, QGuestAllocator *t_alloc) { -QVirtioMMIODevice *dev; -QVirtQueue *vq; -QGuestAllocator *alloc

Re: [Qemu-devel] [PATCH v2 00/34] Qtest driver framework

2018-08-10 Thread Emanuele
On 08/09/2018 11:44 AM, Paolo Bonzini wrote: On 09/08/2018 11:20, Emanuele wrote: Why this? Shouldn't it be: if (g_strcmp0(old_path, path)) {     qtest_end(); /* handles global_qtest = NULL */     g_free(old_path); /* handles NULL */     old_path = path;     global_

Re: [Qemu-devel] [PATCH v2] qpci_free_pc: sdhci-test and vhost-user-test could free() NULL pointers.

2018-07-02 Thread Emanuele
On 07/02/2018 04:35 PM, Paolo Bonzini wrote: On 02/07/2018 16:05, Emanuele Giuseppe Esposito wrote: @@ -152,6 +152,8 @@ QPCIBus *qpci_init_pc(QTestState *qts, QGuestAllocator *alloc) void qpci_free_pc(QPCIBus *bus) { +g_assert(bus); + QPCIBusPC *s = container_of(bus

Re: [Qemu-devel] [PATCH 00/33] Qtest driver framework

2018-08-16 Thread Emanuele
On 15/08/2018 14:38, Markus Armbruster wrote: Emanuele Giuseppe Esposito writes: Qgraph API for the qtest driver framework This series of patches introduce a different qtest driver organization, viewing machines, drivers and tests as node in a graph, each having one or multiple edges

[Bug 1847861] Re: Guest stuttering under high disk IO (virtio)

2019-10-12 Thread Emanuele Faranda
You can find my libvirt XML attached. Here is the full qemu command (taken from the ps output): /usr/bin/qemu-system-x86_64 -name guest=win10,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-7-win10 /master-key.aes -machine pc-q35-3.1,accel=kvm,usb=off,

[Bug 1847861] Re: Guest stuttering under high disk IO (virtio)

2019-10-12 Thread Emanuele Faranda
** Description changed: Performing io intensive tasks on virtualized Windows causes the system to visually stutter. I can often reproduce the problem by running fio on windows: fio --randrepeat=1 --ioengine=windowsaio --direct=1 --gtod_reduce=1 --name=test --filename=\\.\PhysicalDrive

[Bug 1778966] Re: Windows 1803 and later crashes on KVM

2019-10-12 Thread Emanuele Faranda
I ran into the same problem on threadripper 1900X. I was using cpu type "host-passthough" and it crashed. I fixed the crash by disabling the MSR with kvm.ignore_msrs=1 as describe in https://forum.level1techs.com/t/windows-10-1803-as-guest- with-qemu-kvm-bsod-under-install/127425/10 -- You rece

[Bug 1847861] [NEW] Guest stuttering under high disk IO (virtio)

2019-10-12 Thread Emanuele Faranda
Public bug reported: Performing io intensive tasks on virtualized Windows causes the system to visually stutter. I can often reproduce the problem by running fio on windows: fio --randrepeat=1 --ioengine=windowsaio --direct=1 --gtod_reduce=1 --name=test --filename=\\.\PhysicalDrive0 --bs=4k --iod

Re: [PATCH v2 2/9] block-copy: add missing coroutine_fn annotations

2022-11-15 Thread Emanuele Giuseppe Esposito
en in the next few days I send all the rwlock patches. What has been discussed so far (using QEMU_IN_COROUTINE, using some sort of tool to automate everything, etc.) has been noted and as I understand will be researched by Alberto. Thank you, Emanuele Am 10/11/2022 um 11:52 schrieb Paolo Bonzini:

[PATCH v3 1/8] block-copy: add missing coroutine_fn annotations

2022-11-16 Thread Emanuele Giuseppe Esposito
block_copy_reset_unallocated and block_copy_is_cluster_allocated are only called by backup_run, a corotuine_fn itself. Same applies to block_copy_block_status, called by block_copy_dirty_clusters. Therefore mark them as coroutine too. Signed-off-by: Emanuele Giuseppe Esposito --- block/block

[PATCH v3 6/8] block: bdrv_create_file is a coroutine_fn

2022-11-16 Thread Emanuele Giuseppe Esposito
It is always called in coroutine_fn callbacks, therefore it can directly call bdrv_co_create(). Signed-off-by: Emanuele Giuseppe Esposito --- block.c| 6 -- include/block/block-global-state.h | 3 ++- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a

[PATCH v3 5/8] block/vmdk: add missing coroutine_fn annotations

2022-11-16 Thread Emanuele Giuseppe Esposito
vmdk_co_create_opts() is a coroutine_fn, and calls vmdk_co_do_create() which in turn can call two callbacks: vmdk_co_create_opts_cb and vmdk_co_create_cb. Mark all these functions as coroutine_fn, since vmdk_co_create_opts() is the only caller. Signed-off-by: Emanuele Giuseppe Esposito

[PATCH v3 4/8] block: distinguish between bdrv_create running in coroutine and not

2022-11-16 Thread Emanuele Giuseppe Esposito
graph rdlock to the coroutine case. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 74 - 1 file changed, 36 insertions(+), 38 deletions(-) diff --git a/block.c b/block.c index 577639c7e0..375c8056a3 100644 --- a/block.c +++ b

[PATCH v3 7/8] block: bdrv_create is never called in coroutine context

2022-11-16 Thread Emanuele Giuseppe Esposito
Delete the if case and make sure it won't be called again in coroutines. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 37 - 1 file changed, 16 insertions(+), 21 deletions(-) diff --git a/block.c b/block.c index dcac28756c..7a4ce7948c 100644

[PATCH v3 2/8] nbd/server.c: add missing coroutine_fn annotations

2022-11-16 Thread Emanuele Giuseppe Esposito
There are probably more missing, but right now it is necessary that we extend coroutine_fn to block{allock/status}_to_extents, because they use bdrv_* functions calling the generated_co_wrapper API, which checks for the qemu_in_coroutine() case. Signed-off-by: Emanuele Giuseppe Esposito --- nbd

[PATCH v3 3/8] block-backend: replace bdrv_*_above with blk_*_above

2022-11-16 Thread Emanuele Giuseppe Esposito
Avoid mixing bdrv_* functions with blk_*, so create blk_* counterparts for: - bdrv_block_status_above - bdrv_is_allocated_above Note that these functions will take the rdlock, so they must always run in a coroutine. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c

[PATCH v3 0/8] Still more coroutine and various fixes in block layer

2022-11-16 Thread Emanuele Giuseppe Esposito
t; already present in generated_co_wraper functions. - make sure that if a BlockDriver callback is defined as coroutine_fn, then it is always running in a coroutine. This serie is based on Kevin Wolf's series "block: Simplify drain". Based-on: <20221108123738.530873-1-kw...@

[PATCH v3 8/8] block/dirty-bitmap: remove unnecessary qemu_in_coroutine() case

2022-11-16 Thread Emanuele Giuseppe Esposito
Some functions check if they are running in a coroutine, calling the coroutine callback directly if it's the case. Except that no coroutine calls such functions, therefore that case can be removed. Signed-off-by: Emanuele Giuseppe Esposito --- block/dirty-bitmap.c

Re: [PATCH v3 0/8] Still more coroutine and various fixes in block layer

2022-11-16 Thread Emanuele Giuseppe Esposito
I apologize, as discussed also in v2 I just realized I could introduce generated_co_wrapper_simple already here and simplify patches 6 and 8. Also I think commit messages are the old ones from v1. I'll resend. Please ignore this serie. Emanuele Am 16/11/2022 um 09:50 schrieb Emanuele Giu

[PATCH v4 06/11] block-coroutine-wrapper.py: support also basic return types

2022-11-16 Thread Emanuele Giuseppe Esposito
Extend the regex to cover also return type, pointers included. This implies that the value returned by the function cannot be a simple "int" anymore, but the custom return type. Therefore remove poll_state->ret and instead use a per-function custom "ret" field. Signed-of

[PATCH v4 02/11] nbd/server.c: add missing coroutine_fn annotations

2022-11-16 Thread Emanuele Giuseppe Esposito
in other words call qemu_coroutine_yield()). Therefore we need to mark such functions coroutine_fn too. Signed-off-by: Emanuele Giuseppe Esposito --- nbd/server.c | 21 - 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/nbd/server.c b/nbd/server.c index

[PATCH v4 01/11] block-copy: add missing coroutine_fn annotations

2022-11-16 Thread Emanuele Giuseppe Esposito
suspend (or in other words call qemu_coroutine_yield()). Therefore we need to mark such functions coroutine_fn too. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-copy.c | 15 +-- 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/block/block-copy.c b/block/block

[PATCH v4 07/11] block/vmdk: add missing coroutine_fn annotations

2022-11-16 Thread Emanuele Giuseppe Esposito
suspend (or in other words call qemu_coroutine_yield()). Therefore we need to mark such functions coroutine_fn too. Signed-off-by: Emanuele Giuseppe Esposito --- block/vmdk.c | 36 +++- 1 file changed, 19 insertions(+), 17 deletions(-) diff --git a/block/vmdk.c b/block

[PATCH v4 00/11] Still more coroutine and various fixes in block layer

2022-11-16 Thread Emanuele Giuseppe Esposito
t; already present in generated_co_wraper functions. - make sure that if a BlockDriver callback is defined as coroutine_fn, then it is always running in a coroutine. This serie is based on Kevin Wolf's series "block: Simplify drain". Based-on: <20221108123738.530873-1-kw...@

[PATCH v4 04/11] block-coroutine-wrapper.py: introduce generated_co_wrapper_simple

2022-11-16 Thread Emanuele Giuseppe Esposito
functions will be substituted on g_c_w_simple. Signed-off-by: Emanuele Giuseppe Esposito --- include/block/block-common.h | 1 + scripts/block-coroutine-wrapper.py | 87 ++ 2 files changed, 66 insertions(+), 22 deletions(-) diff --git a/include/block/block

[PATCH v4 05/11] block-coroutine-wrapper.py: default to main loop aiocontext if function does not have a BlockDriverState parameter

2022-11-16 Thread Emanuele Giuseppe Esposito
ain loop. Signed-off-by: Emanuele Giuseppe Esposito --- scripts/block-coroutine-wrapper.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/block-coroutine-wrapper.py b/scripts/block-coroutine-wrapper.py index f88ef53964..0f842386d4 100644 --- a/scripts/bl

[PATCH v4 09/11] block: bdrv_create_file is a coroutine_fn

2022-11-16 Thread Emanuele Giuseppe Esposito
It is always called in coroutine_fn callbacks, therefore it can directly call bdrv_co_create(). Signed-off-by: Emanuele Giuseppe Esposito --- block.c| 6 -- include/block/block-global-state.h | 3 ++- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a

[PATCH v4 08/11] block: distinguish between bdrv_create running in coroutine and not

2022-11-16 Thread Emanuele Giuseppe Esposito
Call two different functions depending on whether bdrv_create is in coroutine or not, following the same pattern as generated_co_wrapper functions. This allows to also call the coroutine function directly, without using CreateCo or relying in bdrv_create(). Signed-off-by: Emanuele Giuseppe

[PATCH v4 10/11] block: convert bdrv_create to generated_co_wrapper_simple

2022-11-16 Thread Emanuele Giuseppe Esposito
This function is never called in coroutine context, therefore instead of manually creating a new coroutine, delegate it to the block-coroutine-wrapper script, defining it as g_c_w_simple. Signed-off-by: Emanuele Giuseppe Esposito --- block.c| 38

[PATCH v4 03/11] block-backend: replace bdrv_*_above with blk_*_above

2022-11-16 Thread Emanuele Giuseppe Esposito
Avoid mixing bdrv_* functions with blk_*, so create blk_* counterparts for: - bdrv_block_status_above - bdrv_is_allocated_above Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c | 21 + block/commit.c| 4 ++-- include/sysemu

[PATCH v4 11/11] block/dirty-bitmap: convert coroutine-only functions to generated_co_wrapper_simple

2022-11-16 Thread Emanuele Giuseppe Esposito
ed to g_c_w_simple. Signed-off-by: Emanuele Giuseppe Esposito --- block/dirty-bitmap.c | 88 +--- block/meson.build| 1 + include/block/block-common.h | 5 +- include/block/block-io.h | 9 +++- include/block/dirty-bitmap.h | 10 +++- 5

[PATCH 00/20] Protect the block layer with a rwlock: part 1

2022-11-16 Thread Emanuele Giuseppe Esposito
56527-1-eespo...@redhat.com> Thank you, Emanuele Emanuele Giuseppe Esposito (19): graph-lock: introduce BdrvGraphRWlock structure async: register/unregister aiocontext in graph lock list block.c: wrlock in bdrv_replace_child_noperm block: remove unnecessary assert_bdrv_graph_writable() bl

[PATCH 00/20] Protect the block layer with a rwlock: part 1

2022-11-16 Thread Emanuele Giuseppe Esposito
56527-1-eespo...@redhat.com> Thank you, Emanuele Emanuele Giuseppe Esposito (19): graph-lock: introduce BdrvGraphRWlock structure async: register/unregister aiocontext in graph lock list block.c: wrlock in bdrv_replace_child_noperm block: remove unnecessary assert_bdrv_graph_writable() bl

[PATCH 20/20] block-gen: assert that nbd_co_do_establish_connection is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
The only caller of this function is nbd_do_establish_connection, a generated_co_wrapper that already take the graph read lock. Signed-off-by: Emanuele Giuseppe Esposito --- block/nbd.c | 1 + 1 file changed, 1 insertion(+) diff --git a/block/nbd.c b/block/nbd.c index 7d485c86d2..5cad58aaf6

[PATCH 16/20] block-gen: assert that bdrv_co_{read/write}v_vmstate are always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
-by: Emanuele Giuseppe Esposito --- block/io.c | 2 ++ include/block/block_int-common.h | 2 ++ 2 files changed, 4 insertions(+) diff --git a/block/io.c b/block/io.c index 0bf3919939..c9b451fecd 100644 --- a/block/io.c +++ b/block/io.c @@ -2633,6 +2633,7 @@ bdrv_co_readv_vmst

[PATCH 11/20] block-gen: assert that bdrv_co_{check/invalidate_cache} are always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
The only callers of these functions are the respective generated_co_wrapper, and they already take the lock. Protecting bdrv_co_{check/invalidate_cache}() implies that BlockDriver->bdrv_co_{check/invalidate_cache}() is always called with graph rdlock taken. Signed-off-by: Emanuele Giuse

[PATCH 05/20] block: remove unnecessary assert_bdrv_graph_writable()

2022-11-16 Thread Emanuele Giuseppe Esposito
We don't protect bdrv->aio_context with the graph rwlock, so these assertions are not needed Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/block.c b/block.c index 4ef537a9f2..afab74d4da 100644 --- a/block.c +++ b/block.c @@

[PATCH 17/20] block-gen: assert that bdrv_co_pdiscard is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
BlockDriver callbacks always called with graph rdlock taken: - bdrv_co_pdiscard - bdrv_aio_pdiscard - bdrv_co_pdiscard_snapshot Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c| 1 + block/io.c | 2 ++ include/block/block_int-common.h | 3 +++ 3 files

[PATCH 03/20] async: register/unregister aiocontext in graph lock list

2022-11-16 Thread Emanuele Giuseppe Esposito
Add/remove the AioContext in aio_context_list in graph-lock.c only when it is being effectively created/destroyed. Signed-off-by: Emanuele Giuseppe Esposito --- util/async.c | 4 util/meson.build | 1 + 2 files changed, 5 insertions(+) diff --git a/util/async.c b/util/async.c index

[PATCH 12/20] block-gen: assert that bdrv_co_pwrite is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
: - bdrv_aio_pwritev - bdrv_co_writev - bdrv_co_pwritev - bdrv_co_pwritev_part - bdrv_co_pwritev_compressed - bdrv_co_pwritev_compressed_part Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c| 1 + block/block-copy.c | 8 ++-- block/io.c

[PATCH 02/20] graph-lock: introduce BdrvGraphRWlock structure

2022-11-16 Thread Emanuele Giuseppe Esposito
Just a wrapper to simplify what is available to the struct AioContext. Signed-off-by: Emanuele Giuseppe Esposito --- block/graph-lock.c | 59 ++ include/block/aio.h| 12 include/block/graph-lock.h | 1 + 3 files changed, 48

[PATCH 07/20] graph-lock: implement WITH_GRAPH_RDLOCK_GUARD and GRAPH_RDLOCK_GUARD macros

2022-11-16 Thread Emanuele Giuseppe Esposito
Similar to the implementation in lockable.h, implement macros to automatically take and release the rdlock. Create the empty GraphLockable struct only to use it as a type for G_DEFINE_AUTOPTR_CLEANUP_FUNC. Signed-off-by: Emanuele Giuseppe Esposito --- include/block/graph-lock.h | 35

[PATCH 14/20] block-gen: assert that bdrv_co_pread is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
other generated_co_wrappers: blk_co_pread blk_co_preadv blk_co_preadv_part Protecting bdrv_driver_preadv() implies that the following BlockDriver callbacks always called with graph rdlock taken: - bdrv_co_preadv_part - bdrv_co_preadv - bdrv_aio_preadv - bdrv_co_readv Signed-off-by: Emanuele Giuseppe

[PATCH 15/20] block-gen: assert that {bdrv/blk}_co_flush is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
always called with graph rdlock taken: - bdrv_co_flush - bdrv_co_flush_to_os - bdrv_co_flush_to_disk Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c| 3 ++- block/io.c | 1 + include/block/block_int-common.h | 6 ++ 3 files changed, 9

[PATCH 5/6] block/io: assert that BlockDriver->bdrv_co_*_snapshot_* are always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
The only callers are other callback functions that already run with the graph rdlock taken. Signed-off-by: Emanuele Giuseppe Esposito --- block/io.c | 2 ++ include/block/block_int-common.h | 3 +++ 2 files changed, 5 insertions(+) diff --git a/block/io.c b/block/io.c

[PATCH 09/20] block-backend: introduce new generated_co_wrapper_blk annotation

2022-11-16 Thread Emanuele Giuseppe Esposito
This annotation will be used to distinguish the blk_* API from the bdrv_* API in block-gen.c. The reason for this distinction is that blk_* API eventually result in always calling bdrv_*, which has implications when we introduce the read graph lock. Signed-off-by: Emanuele Giuseppe Esposito

[PATCH 08/20] block-coroutine-wrapper.py: take the graph rdlock in bdrv_* functions

2022-11-16 Thread Emanuele Giuseppe Esposito
assume that the graph rdlock is taken at the coroutine creation, i.e. in g_c_w or in specific coroutines (right now we just consider the g_c_w case). All the blk_co_* are responsible of taking the rdlock (at this point is still a TBD). Suggested-by: Kevin Wolf Signed-off-by: Emanuele Giuseppe

[PATCH 00/20] Protect the block layer with a rwlock: part 1

2022-11-16 Thread Emanuele Giuseppe Esposito
56527-1-eespo...@redhat.com> Thank you, Emanuele Emanuele Giuseppe Esposito (19): graph-lock: introduce BdrvGraphRWlock structure async: register/unregister aiocontext in graph lock list block.c: wrlock in bdrv_replace_child_noperm block: remove unnecessary assert_bdrv_graph_writable() bl

[PATCH 6/6] block: assert that BlockDriver->bdrv_co_delete_file is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
The only callers are other callback functions that already run with the graph rdlock taken. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 1 + include/block/block_int-common.h | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/block.c b

[PATCH 10/20] block-gen: assert that {bdrv/blk}_co_truncate is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
ate() is always called with graph rdlock taken. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c| 1 + block/io.c | 1 + include/block/block_int-common.h | 2 ++ 3 files changed, 4 insertions(+) diff --git a/block/block-backend.c b/block/bl

[PATCH 2/6] block: assert that BlockDriver->bdrv_co_{amend/create} are called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
Both functions are only called by Job->run() callbacks, therefore they must take the lock in the *_run() implementation. Signed-off-by: Emanuele Giuseppe Esposito --- block/amend.c| 1 + block/create.c | 1 + include/block/block_int-common.h | 2 ++ 3 fi

[PATCH 0/6] Protect the block layer with a rwlock: part 2

2022-11-16 Thread Emanuele Giuseppe Esposito
y traverse the BlockDriverState graph, therefore they need to be protected with the rdlock. Based-on: <20221116134850.3051419-1-eespo...@redhat.com> Thank you, Emanuele Emanuele Giuseppe Esposito (6): block: assert that bdrv_co_create is always called with graph rdlock taken bl

[PATCH 06/20] block: assert that graph read and writes are performed correctly

2022-11-16 Thread Emanuele Giuseppe Esposito
Remove the old assert_bdrv_graph_writable, and replace it with the new version using graph-lock API. See the function documentation for more information. Signed-off-by: Emanuele Giuseppe Esposito --- block.c| 4 ++-- block/graph-lock.c | 11

[PATCH 1/6] block: assert that bdrv_co_create is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
ver->bdrv_co_create_opts is always called with graph rdlock taken. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 1 + include/block/block_int-common.h | 1 + 2 files changed, 2 insertions(+) diff --git a/block.c b/block.c index c7611bed9e..e54ed300d7 100644 --- a/block.c

[PATCH 13/20] block-gen: assert that bdrv_co_pwrite_{zeros/sync} is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
Already protected by bdrv_co_pwrite callers. Protecting bdrv_co_do_pwrite_zeroes() implies that BlockDriver->bdrv_co_pwrite_zeroes() is always called with graph rdlock taken. Signed-off-by: Emanuele Giuseppe Esposito --- block/io.c | 3 +++ include/block/block_

[PATCH 19/20] block-gen: assert that bdrv_co_ioctl is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
called too with rdlock taken). Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c| 1 + block/io.c | 1 + include/block/block_int-common.h | 5 +++-- 3 files changed, 5 insertions(+), 2 deletions(-) diff --git a/block/block-backend.c b/block

[PATCH 3/6] block: assert that BlockDriver->bdrv_co_copy_range_{from/to} is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
The only non-protected caller is convert_co_copy_range(), all other callers are BlockDriver callbacks that already take the rdlock. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c| 2 ++ block/io.c | 5 + include/block/block_int-common.h

[PATCH 01/20] block: introduce a lock to protect graph operations

2022-11-16 Thread Emanuele Giuseppe Esposito
we transfer the count to a global shared counter so that the writer is always aware of all readers. Co-developed-with: Emanuele Giuseppe Esposito Signed-off-by: Kevin Wolf Signed-off-by: Paolo Bonzini --- block/graph-lock.c | 221 + block/meson.buil

[PATCH 04/20] block.c: wrlock in bdrv_replace_child_noperm

2022-11-16 Thread Emanuele Giuseppe Esposito
Protect the main function where graph is modified. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 6 -- include/block/block_int-common.h | 1 + 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/block.c b/block.c index d3e168408a..4ef537a9f2

[PATCH 18/20] block-gen: assert that bdrv_co_common_block_status_above is always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
() implies that BlockDriver->bdrv_co_block_status() is always called with graph rdlock taken. Signed-off-by: Emanuele Giuseppe Esposito --- block/backup.c | 3 +++ block/block-backend.c| 2 ++ block/block-copy.c | 2 ++ block/i

[PATCH 4/6] block/dirty-bitmap: assert that BlockDriver->bdrv_co_*_dirty_bitmap are always called with graph rdlock taken

2022-11-16 Thread Emanuele Giuseppe Esposito
The only callers are the respective bdrv_*_dirty_bitmap() functions that take care of creating a new coroutine (that already takes the graph rdlock). Signed-off-by: Emanuele Giuseppe Esposito --- block/dirty-bitmap.c | 2 ++ include/block/block_int-common.h | 2 ++ 2 files changed

[PATCH 13/15] block: convert bdrv_io_plug in generated_co_wrapper_simple

2022-11-16 Thread Emanuele Giuseppe Esposito
k_plug a generated_co_wrapper_simple, so that it always creates a new coroutine, and then make bdrv_plug coroutine_fn. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c | 5 +++-- block/io.c| 5 +++-- include/block/block-io.h | 3 ++- include/block/b

[PATCH 09/15] block-coroutine-wrapper: support void functions

2022-11-16 Thread Emanuele Giuseppe Esposito
Just omit the various 'return' when the return type is void. Signed-off-by: Emanuele Giuseppe Esposito --- scripts/block-coroutine-wrapper.py | 19 ++- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/scripts/block-coroutine-wrapper.py b/scripts/block

[PATCH 12/15] block: convert bdrv_debug_event in generated_co_wrapper

2022-11-16 Thread Emanuele Giuseppe Esposito
ame name. Unfortunately we cannot use a generated_co_wrapper_simple, because the function is called by mixed functions that run both in coroutine and non-coroutine context (for example blkdebug_open). Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 4 +++- bl

[PATCH 00/15] Protect the block layer with a rwlock: part 3

2022-11-16 Thread Emanuele Giuseppe Esposito
s run in a coroutine. Based-on: <20221116135331.3052923-1-eespo...@redhat.com> Thank you, Emanuele Emanuele Giuseppe Esposito (15): block/qed: add missing graph rdlock in qed_need_check_timer_entry block: rename refresh_total_sectors in bdrv_refresh_total_sectors block-backend: use

[PATCH 03/15] block-backend: use bdrv_getlength instead of blk_getlength

2022-11-16 Thread Emanuele Giuseppe Esposito
The only difference is that blk_ checks if the block is available, but this check is already performed above in blk_check_byte_request(). This is in preparation for the graph rdlock, which will be taken by both the callers of blk_check_byte_request() and blk_getlength(). Signed-off-by: Emanuele

[PATCH 04/15] block: convert bdrv_refresh_total_sectors in generated_co_wrapper

2022-11-16 Thread Emanuele Giuseppe Esposito
oContext lock. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 29 - block/block-backend.c | 12 block/commit.c| 4 ++-- block/meson.build | 1 + block/

[PATCH 08/15] block: convert bdrv_is_inserted in generated_co_wrapper_simple

2022-11-16 Thread Emanuele Giuseppe Esposito
ry place it needs to be, we will poll using AIO_WAIT_WHILE_UNLOCKED and remove the AioContext lock. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 5 +++-- block/block-backend.c | 5 +++-- block/io.c| 12 ++-- bl

[PATCH 01/15] block/qed: add missing graph rdlock in qed_need_check_timer_entry

2022-11-16 Thread Emanuele Giuseppe Esposito
graph rdlock. Signed-off-by: Emanuele Giuseppe Esposito --- block/qed.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/qed.c b/block/qed.c index c2691a85b1..778b23d0f6 100644 --- a/block/qed.c +++ b/block/qed.c @@ -282,11 +282,13 @@ static void coroutine_fn

[PATCH 07/15] block: convert bdrv_get_info in generated_co_wrapper

2022-11-16 Thread Emanuele Giuseppe Esposito
ame name. Unfortunately we cannot use a generated_co_wrapper_simple, because the function is called by mixed functions that run both in coroutine and non-coroutine context (for example block_load in migration/block.c). Signed-off-by: Emanuele Giuseppe Esposito --- block.c

[PATCH 06/15] block: convert bdrv_get_allocated_file_size in generated_co_wrapper_simple

2022-11-16 Thread Emanuele Giuseppe Esposito
wrapper with the same name that will take care of always calling the function in a coroutine. This is a generated_co_wrapper_simple callback, which means it assumes that bdrv_get_allocated_file_size is never caled in a coroutine context. Signed-off-by: Emanuele Giuseppe Esposito ---

[PATCH 10/15] block: convert bdrv_eject in generated_co_wrapper_simple

2022-11-16 Thread Emanuele Giuseppe Esposito
_eject a generated_co_wrapper_simple, so that it always creates a new coroutine, and then make bdrv_eject coroutine_fn. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 3 ++- block/block-backend.c | 5 +++-- block/copy-on-read.c | 2 +- block/filter-co

[PATCH 14/15] block: convert bdrv_io_unplug in generated_co_wrapper_simple

2022-11-16 Thread Emanuele Giuseppe Esposito
unplug a generated_co_wrapper_simple, so that it always creates a new coroutine, and then make bdrv_unplug coroutine_fn. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c | 5 +++-- block/io.c| 5 +++-- include/block/block-io.h | 3 +-- inclu

[PATCH 02/15] block: rename refresh_total_sectors in bdrv_refresh_total_sectors

2022-11-16 Thread Emanuele Giuseppe Esposito
Name is not right, since we are going to convert this in a generated_co_wrapper. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 8 block/io.c | 8 +--- include/block/block_int-io.h | 2 +- 3 files

[PATCH 15/15] block: rename newly converted BlockDriver IO coroutine functions

2022-11-16 Thread Emanuele Giuseppe Esposito
Since these functions alwayas run in coroutine context, adjust their name to include "_co_", just like all other BlockDriver callbacks. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 32 ++--- block/

[PATCH 11/15] block: convert bdrv_lock_medium in generated_co_wrapper_simple

2022-11-16 Thread Emanuele Giuseppe Esposito
medium a generated_co_wrapper_simple, so that it always creates a new coroutine, and then make bdrv_lock_medium coroutine_fn. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 3 ++- block/block-backend.c | 5 +++-- block/copy-on-read.c | 2 +

[PATCH 05/15] block: use bdrv_co_refresh_total_sectors when possible

2022-11-16 Thread Emanuele Giuseppe Esposito
In some places we are sure we are always running in a coroutine, therefore it's useless to call the generated_co_wrapper, instead call directly the _co_ function. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c | 6 +++--- block/copy-on-read.c | 2 +- block

Re: [PATCH v3 2/3] KVM: keep track of running ioctls

2022-11-18 Thread Emanuele Giuseppe Esposito
Am 17/11/2022 um 20:27 schrieb David Hildenbrand: > On 11.11.22 16:47, Emanuele Giuseppe Esposito wrote: >> Using the new accel-blocker API, mark where ioctls are being called >> in KVM. Next, we will implement the critical section that will take >> care of performing m

Re: [PATCH 00/15] Protect the block layer with a rwlock: part 3

2022-11-18 Thread Emanuele Giuseppe Esposito
Am 18/11/2022 um 11:57 schrieb Paolo Bonzini: > On 11/16/22 15:07, Emanuele Giuseppe Esposito wrote: >> Here we introduce generated_co_wrapper_simple, a simplification of >> g_c_w that >> only considers the case where the caller is not in a coroutine. >> This simplifi

Re: [PATCH v4 01/11] block-copy: add missing coroutine_fn annotations

2022-11-21 Thread Emanuele Giuseppe Esposito
Am 18/11/2022 um 20:05 schrieb Kevin Wolf: > Am 16.11.2022 um 13:22 hat Emanuele Giuseppe Esposito geschrieben: >> These functions end up calling bdrv_common_block_status_above(), a >> generated_co_wrapper function. >> In addition, they also happen to be always called

Re: [PATCH v4 01/11] block-copy: add missing coroutine_fn annotations

2022-11-21 Thread Emanuele Giuseppe Esposito
Am 21/11/2022 um 09:32 schrieb Emanuele Giuseppe Esposito: > > > Am 18/11/2022 um 20:05 schrieb Kevin Wolf: >> Am 16.11.2022 um 13:22 hat Emanuele Giuseppe Esposito geschrieben: >>> These functions end up calling bdrv_common_block_status_above(), a >>> gene

Re: [PATCH v4 01/11] block-copy: add missing coroutine_fn annotations

2022-11-21 Thread Emanuele Giuseppe Esposito
Am 21/11/2022 um 12:50 schrieb Kevin Wolf: > Am 21.11.2022 um 09:51 hat Emanuele Giuseppe Esposito geschrieben: >> >> >> Am 21/11/2022 um 09:32 schrieb Emanuele Giuseppe Esposito: >>> >>> >>> Am 18/11/2022 um 20:05 schrieb Kevin Wolf: >>&g

Re: [PATCH 00/15] Protect the block layer with a rwlock: part 3

2022-11-21 Thread Emanuele Giuseppe Esposito
rotect the block layer with a rwlock: part 3 Thank you, Emanuele Am 16/11/2022 um 15:07 schrieb Emanuele Giuseppe Esposito: > Please read "Protect the block layer with a rwlock: part 1" and > "Protect the block layer with a rwlock: part 2" for an > additional intro

Re: [PATCH 00/20] Protect the block layer with a rwlock: part 1

2022-11-21 Thread Emanuele Giuseppe Esposito
rotect the block layer with a rwlock: part 3 Thank you, Emanuele Am 16/11/2022 um 14:48 schrieb Emanuele Giuseppe Esposito: > This serie is the first of four series that aim to introduce and use a new > graph rwlock in the QEMU block layer. > The aim is to replace the current AioContext

  1   2   3   4   5   6   7   8   9   10   >