Re: [PATCH v2 11/15] qemu-common: move scripts/qapi

2022-08-22 Thread Marc-André Lureau
Hi On Thu, Aug 11, 2022 at 5:35 PM Markus Armbruster wrote: > Daniel P. Berrangé writes: > > > On Thu, Aug 11, 2022 at 02:50:01PM +0400, Marc-André Lureau wrote: > >> Hi > >> > >> On Thu, Aug 11, 2022 at 2:22 PM Peter Maydell > > >> wrote: > > [...] > > >> > As Markus says, your branch ends up

[PATCH v5 0/9] parallels: Refactor the code of images checks and fix a bug

2022-08-22 Thread Alexander Ivanov
Fix image inflation when offset in BAT is out of image. Replace whole BAT syncing by flushing only dirty blocks. Move all the checks outside the main check function in separate functions Use WITH_QEMU_LOCK_GUARD for simplier code. v5: 2: Change the way of data_end fixing. 6,7: Move data_end ch

[PATCH v5 1/9] parallels: Out of image offset in BAT leads to image inflation

2022-08-22 Thread Alexander Ivanov
data_end field in BDRVParallelsState is set to the biggest offset present in BAT. If this offset is outside of the image, any further write will create the cluster at this offset and/or the image will be truncated to this offset on close. This is definitely not correct. Raise an error in parallels_

[PATCH v5 3/9] parallels: create parallels_set_bat_entry_helper() to assign BAT value

2022-08-22 Thread Alexander Ivanov
This helper will be reused in next patches during parallels_co_check rework to simplify its code. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/parallels.c | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff

[PATCH v5 4/9] parallels: Use generic infrastructure for BAT writing in parallels_co_check()

2022-08-22 Thread Alexander Ivanov
BAT is written in the context of conventional operations over the image inside bdrv_co_flush() when it calls parallels_co_flush_to_os() callback. Thus we should not modify BAT array directly, but call parallels_set_bat_entry() helper and bdrv_co_flush() further on. After that there is no need to ma

[PATCH v5 6/9] parallels: Move check of cluster outside image to a separate function

2022-08-22 Thread Alexander Ivanov
We will add more and more checks so we need a better code structure in parallels_co_check. Let each check performs in a separate loop in a separate helper. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/parallels.c | 50

[PATCH v5 5/9] parallels: Move check of unclean image to a separate function

2022-08-22 Thread Alexander Ivanov
We will add more and more checks so we need a better code structure in parallels_co_check. Let each check performs in a separate loop in a separate helper. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/parallels.c | 31

[PATCH v5 7/9] parallels: Move check of leaks to a separate function

2022-08-22 Thread Alexander Ivanov
We will add more and more checks so we need a better code structure in parallels_co_check. Let each check performs in a separate loop in a separate helper. Signed-off-by: Alexander Ivanov --- block/parallels.c | 87 +-- 1 file changed, 54 insertions(+)

Re: [RFC PATCH] qemu-options: try and clarify preferred block semantics

2022-08-22 Thread Kevin Wolf
Am 19.08.2022 um 19:47 hat Alex Bennée geschrieben: > Try to correct any confusion about QEMU's Byzantine disk options by > laying out the preferred "modern" options as-per: > > " (best: -device + -blockdev, 2nd obsolete syntax: -device + > -drive, 3rd obsolete syntax: -drive, 4th obsolet

[PATCH v5 9/9] parallels: Replace qemu_co_mutex_lock by WITH_QEMU_LOCK_GUARD

2022-08-22 Thread Alexander Ivanov
Replace the way we use mutex in parallels_co_check() for simplier and less error prone code. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev --- block/parallels.c | 33 ++--- 1 file changed, 14 insertions(+), 19 deletions(-) diff --git a/block/parallels

[PATCH v5 8/9] parallels: Move statistic collection to a separate function

2022-08-22 Thread Alexander Ivanov
We will add more and more checks so we need a better code structure in parallels_co_check. Let each check performs in a separate loop in a separate helper. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/parallels.c | 53

[PATCH v5 2/9] parallels: Fix data_end field value in parallels_co_check()

2022-08-22 Thread Alexander Ivanov
Make data_end pointing to the end of the last cluster if a leak was fixed. Otherwise set the file size to data_end. Signed-off-by: Alexander Ivanov --- block/parallels.c | 10 +- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/block/parallels.c b/block/parallels.c index c245

[PATCH v3 0/2] block: add missed block_acct_setup with new block device init procedure

2022-08-22 Thread Denis V. Lunev
Commit 5f76a7aac156ca75680dad5df4a385fd0b58f6b1 is looking harmless from the first glance, but it has changed things a lot. 'libvirt' uses it to detect that it should follow new initialization way and this changes things considerably. With this procedure followed, blockdev_init() is not called anym

[PATCH 1/2] target/hppa: Fix proberi instruction emulation for linux-user

2022-08-22 Thread Denis V. Lunev
From: Helge Deller The proberi assembler instruction checks the read/write access rights for the page of a given address and shall return a value of 1 if the test succeeds and a value of 0 on failure in the target register. But when run in linux-user mode, qemu currently simply returns the retur

[PATCH 2/2] block: pass OnOffAuto instead of bool to block_acct_setup()

2022-08-22 Thread Denis V. Lunev
We would have one more place for block_acct_setup() calling, which should not corrupt original value. Signed-off-by: Denis V. Lunev CC: Peter Krempa CC: Markus Armbruster CC: John Snow CC: Kevin Wolf CC: Hanna Reitz CC: Vladimir Sementsov-Ogievskiy --- block/accounting.c | 26 +

Re: [RFC PATCH] qemu-options: try and clarify preferred block semantics

2022-08-22 Thread Daniel P . Berrangé
On Fri, Aug 19, 2022 at 06:47:57PM +0100, Alex Bennée wrote: > Try to correct any confusion about QEMU's Byzantine disk options by > laying out the preferred "modern" options as-per: > > " (best: -device + -blockdev, 2nd obsolete syntax: -device + > -drive, 3rd obsolete syntax: -drive, 4t

Re: [PATCH v3 0/2] block: add missed block_acct_setup with new block device init procedure

2022-08-22 Thread Denis V. Lunev
On 22.08.2022 12:31, Denis V. Lunev wrote: Commit 5f76a7aac156ca75680dad5df4a385fd0b58f6b1 is looking harmless from the first glance, but it has changed things a lot. 'libvirt' uses it to detect that it should follow new initialization way and this changes things considerably. With this procedure

Re: [RFC PATCH] qemu-options: try and clarify preferred block semantics

2022-08-22 Thread Peter Maydell
On Fri, 19 Aug 2022 at 19:10, Alex Bennée wrote: > > Try to correct any confusion about QEMU's Byzantine disk options by > laying out the preferred "modern" options as-per: > > " (best: -device + -blockdev, 2nd obsolete syntax: -device + > -drive, 3rd obsolete syntax: -drive, 4th obsolete

[PATCH for-7.1?] scsi-generic: Fix emulated block limits VPD page

2022-08-22 Thread Kevin Wolf
Commits 01ef8185b80 amd 24b36e9813e updated the way that the maximum transfer length is calculated for patching block limits VPD page in an INQUIRY response. The same updates also need to be made for the case where the host device does not support the block limits VPD page at all and we emulate th

Re: [PATCH 1/2] target/hppa: Fix proberi instruction emulation for linux-user

2022-08-22 Thread Helge Deller
Thank you Denis, Btw, this patch was already pulled by Richard... Helge On 8/22/22 12:31, Denis V. Lunev wrote: > From: Helge Deller > > The proberi assembler instruction checks the read/write access rights > for the page of a given address and shall return a value of 1 if the > test succeeds a

[PATCH v1 6/6] qemu-options: try and clarify preferred block semantics

2022-08-22 Thread Alex Bennée
Try to correct any confusion about QEMU's Byzantine disk options by laying out the preferred "modern" options as-per: " (best: -device + -blockdev, 2nd obsolete syntax: -device + -drive, 3rd obsolete syntax: -drive, 4th obsolete syntax: -hdNN)" Signed-off-by: Alex Bennée Acked-by: Kevin

[RFC v4 01/11] blkio: add libblkio block driver

2022-08-22 Thread Stefan Hajnoczi
libblkio (https://gitlab.com/libblkio/libblkio/) is a library for high-performance disk I/O. It currently supports io_uring, virtio-blk-vhost-user, and virtio-blk-vhost-vdpa with additional drivers under development. One of the reasons for developing libblkio is that other applications besides QEM

[RFC v4 00/11] blkio: add libblkio BlockDriver

2022-08-22 Thread Stefan Hajnoczi
v4: - Patch 1: - Add virtio-blk-vhost-user driver [Kevin] - Drop .bdrv_parse_filename() and .bdrv_needs_filename for virtio-blk-vhost-vdpa [Stefano] - Add copyright and license header [Hanna] - Drop .bdrv_parse_filename() in favor of --blockdev or json: [Hanna] - Clarify that "filename"

[RFC v4 04/11] block: use BdrvRequestFlags type for supported flag fields

2022-08-22 Thread Stefan Hajnoczi
Use the enum type so GDB displays the enum members instead of printing a numeric constant. Signed-off-by: Stefan Hajnoczi --- include/block/block_int-common.h | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/include/block/block_int-common.h b/include/block/block_int-co

[RFC v4 02/11] numa: call ->ram_block_removed() in ram_block_notifer_remove()

2022-08-22 Thread Stefan Hajnoczi
When a RAMBlockNotifier is added, ->ram_block_added() is called with all existing RAMBlocks. There is no equivalent ->ram_block_removed() call when a RAMBlockNotifier is removed. The util/vfio-helpers.c code (the sole user of RAMBlockNotifier) is fine with this asymmetry because it does not rely o

[RFC v4 06/11] block: return errors from bdrv_register_buf()

2022-08-22 Thread Stefan Hajnoczi
Registering an I/O buffer is only a performance optimization hint but it is still necessary to return errors when it fails. Later patches will need to detect errors when registering buffers but an immediate advantage is that error_report() calls are no longer needed in block driver .bdrv_register_

[RFC v4 08/11] exec/cpu-common: add qemu_ram_get_fd()

2022-08-22 Thread Stefan Hajnoczi
Add a function to get the file descriptor for a RAMBlock. Device emulation code typically uses the MemoryRegion APIs but vhost-style code may use RAMBlock directly for sharing guest memory with another process. This new API will be used by the libblkio block driver so it can share guest memory via

[RFC v4 03/11] block: pass size to bdrv_unregister_buf()

2022-08-22 Thread Stefan Hajnoczi
The only implementor of bdrv_register_buf() is block/nvme.c, where the size is not needed when unregistering a buffer. This is because util/vfio-helpers.c can look up mappings by address. Future block drivers that implement bdrv_register_buf() may not be able to do their job given only the buffer

[RFC v4 09/11] stubs: add qemu_ram_block_from_host() and qemu_ram_get_fd()

2022-08-22 Thread Stefan Hajnoczi
The blkio block driver will need to look up the file descriptor for a given pointer. This is possible in softmmu builds where the RAMBlock API is available for querying guest RAM. Add stubs so tools like qemu-img that link the block layer still build successfully. In this case there is no guest RA

[RFC v4 05/11] block: add BDRV_REQ_REGISTERED_BUF request flag

2022-08-22 Thread Stefan Hajnoczi
Block drivers may optimize I/O requests accessing buffers previously registered with bdrv_register_buf(). Checking whether all elements of a request's QEMUIOVector are within previously registered buffers is expensive, so we need a hint from the user to avoid costly checks. Add a BDRV_REQ_REGISTER

[RFC v4 07/11] block: add BlockRAMRegistrar

2022-08-22 Thread Stefan Hajnoczi
Emulated devices and other BlockBackend users wishing to take advantage of blk_register_buf() all have the same repetitive job: register RAMBlocks with the BlockBackend using RAMBlockNotifier. Add a BlockRAMRegistrar API to do this. A later commit will use this from hw/block/virtio-blk.c. Signed-

[RFC v4 10/11] blkio: implement BDRV_REQ_REGISTERED_BUF optimization

2022-08-22 Thread Stefan Hajnoczi
Avoid bounce buffers when QEMUIOVector elements are within previously registered bdrv_register_buf() buffers. The idea is that emulated storage controllers will register guest RAM using bdrv_register_buf() and set the BDRV_REQ_REGISTERED_BUF on I/O requests. Therefore no blkio_map_mem_region() cal

[RFC v4 11/11] virtio-blk: use BDRV_REQ_REGISTERED_BUF optimization hint

2022-08-22 Thread Stefan Hajnoczi
Register guest RAM using BlockRAMRegistrar and set the BDRV_REQ_REGISTERED_BUF flag so block drivers can optimize memory accesses in I/O requests. This is for vdpa-blk, vhost-user-blk, and other I/O interfaces that rely on DMA mapping/unmapping. Signed-off-by: Stefan Hajnoczi --- include/hw/vir

Re: [PATCH v7 2/8] file-posix: introduce get_sysfs_str_val for device zoned model

2022-08-22 Thread Stefan Hajnoczi
On Tue, Aug 16, 2022 at 02:25:16PM +0800, Sam Li wrote: > +static int hdev_get_max_segments(int fd, struct stat *st) { > +int ret; > +if (S_ISCHR(st->st_mode)) { > +if (ioctl(fd, SG_GET_SG_TABLESIZE, &ret) == 0) { The ioctl must be within #ifdef CONFIG_LINUX since SG_GET_SG_TABLESI

Re: [PATCH v7 4/8] block: add block layer APIs resembling Linux ZonedBlockDevice ioctls

2022-08-22 Thread Stefan Hajnoczi
On Tue, Aug 16, 2022 at 02:25:18PM +0800, Sam Li wrote: > By adding zone management operations in BlockDriver, storage controller > emulation can use the new block layer APIs including Report Zone and > four zone management operations (open, close, finish, reset). > > Add zoned storage commands of

Re: [PATCH v7 6/8] config: add check to block layer

2022-08-22 Thread Stefan Hajnoczi
On Tue, Aug 16, 2022 at 02:25:20PM +0800, Sam Li wrote: > Putting zoned/non-zoned BlockDrivers on top of each other is not > allowed. > > Signed-off-by: Sam Li > Reviewed-by: Stefan Hajnoczi > --- > block.c | 14 ++ > block/raw-format.c | 1 +

Re: [PATCH v7 4/8] block: add block layer APIs resembling Linux ZonedBlockDevice ioctls

2022-08-22 Thread Sam Li
Stefan Hajnoczi 于2022年8月23日周二 08:49写道: > > On Tue, Aug 16, 2022 at 02:25:18PM +0800, Sam Li wrote: > > By adding zone management operations in BlockDriver, storage controller > > emulation can use the new block layer APIs including Report Zone and > > four zone management operations (open, close,

Re: [PATCH v7 6/8] config: add check to block layer

2022-08-22 Thread Sam Li
Stefan Hajnoczi 于2022年8月23日周二 08:54写道: > > On Tue, Aug 16, 2022 at 02:25:20PM +0800, Sam Li wrote: > > Putting zoned/non-zoned BlockDrivers on top of each other is not > > allowed. > > > > Signed-off-by: Sam Li > > Reviewed-by: Stefan Hajnoczi > > --- > > block.c | 14 +

Re: [PATCH v7 2/8] file-posix: introduce get_sysfs_str_val for device zoned model

2022-08-22 Thread Sam Li
Stefan Hajnoczi 于2022年8月23日周二 07:05写道: > > On Tue, Aug 16, 2022 at 02:25:16PM +0800, Sam Li wrote: > > +static int hdev_get_max_segments(int fd, struct stat *st) { > > +int ret; > > +if (S_ISCHR(st->st_mode)) { > > +if (ioctl(fd, SG_GET_SG_TABLESIZE, &ret) == 0) { > > The ioctl mus

Re: [PATCH v5 1/9] parallels: Out of image offset in BAT leads to image inflation

2022-08-22 Thread Vladimir Sementsov-Ogievskiy
On 8/22/22 12:05, Alexander Ivanov wrote: data_end field in BDRVParallelsState is set to the biggest offset present in BAT. If this offset is outside of the image, any further write will create the cluster at this offset and/or the image will be truncated to this offset on close. This is definite