[PATCH v6 0/8] Add support for zoned device

2022-08-05 Thread Sam Li
Zoned Block Devices (ZBDs) devide the LBA space to block regions called zones that are larger than the LBA size. It can only allow sequential writes, which reduces write amplification in SSD, leading to higher throughput and increased capacity. More details about ZBDs can be found at: https://zone

[PATCH v6 1/8] include: add zoned device structs

2022-08-05 Thread Sam Li
Signed-off-by: Sam Li Reviewed-by: Stefan Hajnoczi --- include/block/block-common.h | 43 1 file changed, 43 insertions(+) diff --git a/include/block/block-common.h b/include/block/block-common.h index fdb7306e78..36bd0e480e 100644 --- a/include/block/block-

[PATCH v6 2/8] file-posix: introduce get_sysfs_long_val for the long sysfs attribute

2022-08-05 Thread Sam Li
Use sysfs attribute files to get the long value of zoned device information. Signed-off-by: Sam Li Reviewed-by: Hannes Reinecke --- block/file-posix.c | 37 +++-- 1 file changed, 23 insertions(+), 14 deletions(-) diff --git a/block/file-posix.c b/block/file-posi

[PATCH v6 6/8] config: add check to block layer

2022-08-05 Thread Sam Li
Putting zoned/non-zoned BlockDrivers on top of each other is not allowed. Signed-off-by: Sam Li Reviewed-by: Stefan Hajnoczi --- block.c | 13 + block/file-posix.c | 1 + block/raw-format.c | 1 + include/block/block_int-common.

[PATCH v6 4/8] block: add block layer APIs resembling Linux ZonedBlockDevice ioctls

2022-08-05 Thread Sam Li
By adding zone management operations in BlockDriver, storage controller emulation can use the new block layer APIs including Report Zone and four zone management operations (open, close, finish, reset). Add zoned storage commands of the device: zone_report(zrp), zone_open(zo), zone_close(zc), zone

[PATCH v6 8/8] docs/zoned-storage: add zoned device documentation

2022-08-05 Thread Sam Li
Add the documentation about the zoned device support to virtio-blk emulation. Signed-off-by: Sam Li --- docs/devel/zoned-storage.rst | 41 ++ docs/system/qemu-block-drivers.rst.inc | 6 2 files changed, 47 insertions(+) create mode 100644 docs/devel/zoned

[PATCH v6 3/8] file-posix: introduce get_sysfs_str_val for device zoned model

2022-08-05 Thread Sam Li
Use sysfs attribute files to get the string value of device zoned model. Then get_sysfs_zoned_model can convert it to BlockZoneModel type in QEMU. Signed-off-by: Sam Li Reviewed-by: Hannes Reinecke --- block/file-posix.c | 70 include/block/block_i

[PATCH v6 5/8] raw-format: add zone operations to pass through requests

2022-08-05 Thread Sam Li
raw-format driver usually sits on top of file-posix driver. It needs to pass through requests of zone commands. Signed-off-by: Sam Li Reviewed-by: Stefan Hajnoczi --- block/raw-format.c | 13 + 1 file changed, 13 insertions(+) diff --git a/block/raw-format.c b/block/raw-format.c in

[PATCH v6 7/8] qemu-iotests: test new zone operations

2022-08-05 Thread Sam Li
We have added new block layer APIs of zoned block devices. Test it with: Create a null_blk device, run each zone operation on it and see whether reporting right zone information. Signed-off-by: Sam Li Reviewed-by: Stefan Hajnoczi --- tests/qemu-iotests/tests/zoned.out | 53 ++ t

Re: [PATCH v2 11/15] qemu-common: move scripts/qapi

2022-08-05 Thread Markus Armbruster
marcandre.lur...@redhat.com writes: > From: Marc-André Lureau > > This is just moving qapi-gen.py and related subdir to qemu-common, to > ease review and proceed step by step. The following patches will move > related necessary code, tests etc. > > Signed-off-by: Marc-André Lureau As moved file

Re: [PATCH v10 12/21] commit and mirror: create new nodes using bdrv_get_aio_context, and not the job aiocontext

2022-08-05 Thread Kevin Wolf
Am 25.07.2022 um 09:38 hat Emanuele Giuseppe Esposito geschrieben: > We are always using the given bs AioContext, so there is no need > to take the job ones (which is identical anyways). > This also reduces the point we need to check when protecting > job.aio_context field. > > Reviewed-by: Vladim

Re: [PATCH v10 13/21] job: detect change of aiocontext within job coroutine

2022-08-05 Thread Kevin Wolf
Am 25.07.2022 um 09:38 hat Emanuele Giuseppe Esposito geschrieben: > From: Paolo Bonzini > > We want to make sure access of job->aio_context is always done > under either BQL or job_mutex. Is this the goal of this series? If so, it would have been useful to state somewhere more obvious, because

Re: [PATCH v2 11/15] qemu-common: move scripts/qapi

2022-08-05 Thread Marc-André Lureau
Hi On Fri, Aug 5, 2022 at 12:12 PM Markus Armbruster wrote: > marcandre.lur...@redhat.com writes: > > > From: Marc-André Lureau > > > > This is just moving qapi-gen.py and related subdir to qemu-common, to > > ease review and proceed step by step. The following patches will move > > related nec

Re: [PATCH v10 14/21] jobs: protect job.aio_context with BQL and job_mutex

2022-08-05 Thread Kevin Wolf
Am 25.07.2022 um 09:38 hat Emanuele Giuseppe Esposito geschrieben: > In order to make it thread safe, implement a "fake rwlock", > where we allow reads under BQL *or* job_mutex held, but > writes only under BQL *and* job_mutex. Oh, so the "or BQL" part is only for job.aio_context? Okay. > The onl

Re: [PATCH v10 15/21] blockjob.h: categorize fields in struct BlockJob

2022-08-05 Thread Kevin Wolf
Am 25.07.2022 um 09:38 hat Emanuele Giuseppe Esposito geschrieben: > The same job lock is being used also to protect some of blockjob fields. > Categorize them just as done in job.h. > > Reviewed-by: Vladimir Sementsov-Ogievskiy > Signed-off-by: Emanuele Giuseppe Esposito Why do speed, limit an

Re: [PATCH v10 16/21] blockjob: rename notifier callbacks as _locked

2022-08-05 Thread Kevin Wolf
Am 25.07.2022 um 09:38 hat Emanuele Giuseppe Esposito geschrieben: > They all are called with job_lock held, in job_event_*_locked() Worth documenting that for the notifier lists in struct Job, too? > Reviewed-by: Vladimir Sementsov-Ogievskiy > Reviewed-by: Stefan Hajnoczi > Signed-off-by: Eman

Re: [PATCH v2 14/15] mtest2make.py: teach suite name that are just "PROJECT"

2022-08-05 Thread Paolo Bonzini
On 7/12/22 11:35, marcandre.lur...@redhat.com wrote: From: Marc-André Lureau A subproject test may be simply in the "PROJECT" suite (such as "qemu-common" with the following patches) Signed-off-by: Marc-André Lureau --- scripts/mtest2make.py | 7 +-- 1 file changed, 5 insertions(+), 2

Re: [PATCH v10 17/21] blockjob: protect iostatus field in BlockJob struct

2022-08-05 Thread Kevin Wolf
Am 25.07.2022 um 09:38 hat Emanuele Giuseppe Esposito geschrieben: > iostatus is the only field (together with .job) that needs > protection using the job mutex. > > It is set in the main loop (GLOBAL_STATE functions) but read > in I/O code (block_job_error_action). > > In order to protect it, ch

Re: [PATCH v2 14/15] mtest2make.py: teach suite name that are just "PROJECT"

2022-08-05 Thread Marc-André Lureau
Hi On Fri, Aug 5, 2022 at 2:39 PM Paolo Bonzini wrote: > On 7/12/22 11:35, marcandre.lur...@redhat.com wrote: > > From: Marc-André Lureau > > > > A subproject test may be simply in the "PROJECT" suite (such as > > "qemu-common" with the following patches) > > > > Signed-off-by: Marc-André Lurea

Re: [PATCH v10 18/21] job.c: enable job lock/unlock and remove Aiocontext locks

2022-08-05 Thread Kevin Wolf
Am 25.07.2022 um 09:38 hat Emanuele Giuseppe Esposito geschrieben: > Change the job_{lock/unlock} and macros to use job_mutex. > > Now that they are not nop anymore, remove the aiocontext > to avoid deadlocks. Okay, so this is the big bad patch where we need to verify the completeness of all chan

Re: [PATCH v10 19/21] block_job_query: remove atomic read

2022-08-05 Thread Kevin Wolf
Am 25.07.2022 um 09:38 hat Emanuele Giuseppe Esposito geschrieben: > Not sure what the atomic here was supposed to do, since job.busy > is protected by the job lock. Since the whole function > is called under job_mutex, just remove the atomic. > > Reviewed-by: Vladimir Sementsov-Ogievskiy > Revie

Re: [PATCH v10 21/21] job: remove unused functions

2022-08-05 Thread Kevin Wolf
Am 25.07.2022 um 09:38 hat Emanuele Giuseppe Esposito geschrieben: > These public functions are not used anywhere, thus can be dropped. > Also, since this is the final job API that doesn't use AioContext > lock and replaces it with job_lock, adjust all remaining function > documentation to clearly

Re: [PATCH v10 20/21] blockjob: remove unused functions

2022-08-05 Thread Kevin Wolf
Am 25.07.2022 um 09:38 hat Emanuele Giuseppe Esposito geschrieben: > These public functions are not used anywhere, thus can be dropped. > > Signed-off-by: Emanuele Giuseppe Esposito > Reviewed-by: Stefan Hajnoczi > @@ -113,6 +111,7 @@ BlockJob *block_job_next_locked(BlockJob *job); > * Get th

Re: [PATCH v2 00/11] Refactor bdrv_try_set_aio_context using transactions

2022-08-05 Thread Emanuele Giuseppe Esposito
Am 27/07/2022 um 18:13 schrieb Vladimir Sementsov-Ogievskiy: > On 7/25/22 15:21, Emanuele Giuseppe Esposito wrote: >> The aim of this series is to reorganize bdrv_try_set_aio_context >> and drop BDS ->set_aio_context and ->can_set_aio_ctx callbacks in >> favour of a new one, ->change_aio_ctx. >>

Re: [PATCH v2 00/11] Refactor bdrv_try_set_aio_context using transactions

2022-08-05 Thread Emanuele Giuseppe Esposito
Am 05/08/2022 um 15:22 schrieb Emanuele Giuseppe Esposito: > > > Am 27/07/2022 um 18:13 schrieb Vladimir Sementsov-Ogievskiy: >> On 7/25/22 15:21, Emanuele Giuseppe Esposito wrote: >>> The aim of this series is to reorganize bdrv_try_set_aio_context >>> and drop BDS ->set_aio_context and ->can

Re: [PATCH v2 00/11] Refactor bdrv_try_set_aio_context using transactions

2022-08-05 Thread Vladimir Sementsov-Ogievskiy
On 8/5/22 16:22, Emanuele Giuseppe Esposito wrote: Am 27/07/2022 um 18:13 schrieb Vladimir Sementsov-Ogievskiy: On 7/25/22 15:21, Emanuele Giuseppe Esposito wrote: The aim of this series is to reorganize bdrv_try_set_aio_context and drop BDS ->set_aio_context and ->can_set_aio_ctx callbacks i

[PATCH 2/2] util/aio-win32: Correct the event array size in aio_poll()

2022-08-05 Thread Bin Meng
From: Bin Meng WaitForMultipleObjects() can only wait for MAXIMUM_WAIT_OBJECTS object handles. Correct the event array size in aio_poll() and add a assert() to ensure it does not cause out of bound access. Signed-off-by: Bin Meng --- util/aio-win32.c | 3 ++- 1 file changed, 2 insertions(+),

Re: [PATCH v2 00/11] Refactor bdrv_try_set_aio_context using transactions

2022-08-05 Thread Emanuele Giuseppe Esposito
Am 05/08/2022 um 16:35 schrieb Vladimir Sementsov-Ogievskiy: > On 8/5/22 16:22, Emanuele Giuseppe Esposito wrote: >> >> >> Am 27/07/2022 um 18:13 schrieb Vladimir Sementsov-Ogievskiy: >>> On 7/25/22 15:21, Emanuele Giuseppe Esposito wrote: The aim of this series is to reorganize bdrv_try_se

Re: [PATCH 2/2] util/aio-win32: Correct the event array size in aio_poll()

2022-08-05 Thread Stefan Weil via
Am 05.08.22 um 16:56 schrieb Bin Meng: From: Bin Meng WaitForMultipleObjects() can only wait for MAXIMUM_WAIT_OBJECTS object handles. Correct the event array size in aio_poll() and add a assert() to ensure it does not cause out of bound access. Signed-off-by: Bin Meng --- util/aio-win32.c

Re: [PATCH v2 00/11] Refactor bdrv_try_set_aio_context using transactions

2022-08-05 Thread Vladimir Sementsov-Ogievskiy
On 8/5/22 16:36, Emanuele Giuseppe Esposito wrote: Am 05/08/2022 um 15:22 schrieb Emanuele Giuseppe Esposito: Am 27/07/2022 um 18:13 schrieb Vladimir Sementsov-Ogievskiy: On 7/25/22 15:21, Emanuele Giuseppe Esposito wrote: The aim of this series is to reorganize bdrv_try_set_aio_context an

[PATCH v2 1/3] parallels: Put the image checks in separate functions

2022-08-05 Thread alexander . ivanov
From: Alexander Ivanov We will add more and more checks of images so we need to reorganize the code. Put each check to a separate helper function with a separate loop. Add two helpers: truncate_file() and sync_header(). They will be used in multiple functions. Signed-off-by: Alexander Ivanov --

[PATCH v2 0/3] Check and repair duplicated clusters in parallels images

2022-08-05 Thread alexander . ivanov
From: Alexander Ivanov We will add more and more checks of images so we need to reorganize the code. Put each check to a separate helper function with a separate loop. Add two helpers: truncate_file() and sync_header(). They will be used in multiple functions. Parallels image file can be corrupt

[PATCH v2 2/3] parallels: Add checking and repairing duplicate offsets in BAT

2022-08-05 Thread alexander . ivanov
From: Alexander Ivanov There could be corruptions in the image file: two guest memory areas refer to the same host cluster. If a duplicate offset is found fix it by copying the content of the referred cluster to a new allocated cluster and replace one of the two referring entries by the new clus

[PATCH v2 3/3] iotests, parallels: Add a test for duplicated clusters

2022-08-05 Thread alexander . ivanov
From: Alexander Ivanov Check if original and duplicated offsets refer to the same cluster. Repair the image and check that writing to a referred cluster doesn't affects another referred cluster. Signed-off-by: Natalia Kuzmina Signed-off-by: Alexander Ivanov --- tests/qemu-iotests/314