Re: [RFC 0/8] Introduce an extensible static analyzer

2022-07-05 Thread Daniel P . Berrangé
On Mon, Jul 04, 2022 at 08:30:08PM +0100, Alberto Faria wrote: > On Mon, Jul 4, 2022 at 5:28 PM Daniel P. Berrangé wrote: > > Have you done any measurement see how much of the overhead is from > > the checks you implemented, vs how much is inherantly forced on us > > by libclang ? ie what does it

Re: [PATCH v8 02/20] job.h: categorize fields in struct Job

2022-07-05 Thread Stefan Hajnoczi
On Wed, Jun 29, 2022 at 10:15:20AM -0400, Emanuele Giuseppe Esposito wrote: > Categorize the fields in struct Job to understand which ones > need to be protected by the job mutex and which don't. > > Signed-off-by: Emanuele Giuseppe Esposito > Reviewed-by: Vladimir Sementsov-Ogievskiy > --- > i

Re: [PATCH v8 05/20] job.c: add job_lock/unlock while keeping job.h intact

2022-07-05 Thread Stefan Hajnoczi
On Wed, Jun 29, 2022 at 10:15:23AM -0400, Emanuele Giuseppe Esposito wrote: > With "intact" we mean that all job.h functions implicitly > take the lock. Therefore API callers are unmodified. > > This means that: > - all static functions become _locked, and call _locked functions > - all public fun

Re: [PATCH v8 05/20] job.c: add job_lock/unlock while keeping job.h intact

2022-07-05 Thread Stefan Hajnoczi
On Wed, Jun 29, 2022 at 10:15:23AM -0400, Emanuele Giuseppe Esposito wrote: > +void job_ref(Job *job) > +{ > +JOB_LOCK_GUARD(); > +job_ref_locked(job); > +} You don't need to fix this, but just a note: This API seems dangerous. If we don't hold the lock, how can we be sure job won't be un

Re: [PATCH v8 06/20] job.h: define functions called without job lock held

2022-07-05 Thread Stefan Hajnoczi
On Wed, Jun 29, 2022 at 10:15:24AM -0400, Emanuele Giuseppe Esposito wrote: > These functions don't need a _locked() counterpart, since > they are all called outside job.c and take the lock only > internally. > > Update also the comments in blockjob.c (and move them in job.c). > > Note: at this s

Re: [PATCH v8 07/20] job.h: add _locked public functions

2022-07-05 Thread Stefan Hajnoczi
On Wed, Jun 29, 2022 at 10:15:25AM -0400, Emanuele Giuseppe Esposito wrote: > These functions will be used later when we use the job lock. > > Note: at this stage, job_{lock/unlock} and job lock guard macros > are *nop*. > > Signed-off-by: Emanuele Giuseppe Esposito > --- > include/qemu/job.h |

Re: [PATCH 07/18] block: Add blk_{preadv,pwritev}()

2022-07-05 Thread Hanna Reitz
On 17.05.22 13:38, Alberto Faria wrote: Implement them using generated_co_wrapper. Signed-off-by: Alberto Faria --- include/sysemu/block-backend-io.h | 6 + tests/unit/test-block-iothread.c | 42 ++- 2 files changed, 47 insertions(+), 1 deletion(-) Review

Re: [PATCH v8 08/20] blockjob.h: introduce block_job _locked() APIs

2022-07-05 Thread Stefan Hajnoczi
On Wed, Jun 29, 2022 at 10:15:26AM -0400, Emanuele Giuseppe Esposito wrote: > +BlockJob *block_job_next(BlockJob *bjob) > { > -Job *job = job_get(id); > +JOB_LOCK_GUARD(); > +return block_job_next_locked(bjob); > +} This seems unsafe for the same reason as job_ref(). How can the calle

Re: [PATCH v8 10/20] jobs: add job lock in find_* functions

2022-07-05 Thread Stefan Hajnoczi
On Wed, Jun 29, 2022 at 10:15:28AM -0400, Emanuele Giuseppe Esposito wrote: > Both blockdev.c and job-qmp.c have TOC/TOU conditions, because > they first search for the job and then perform an action on it. > Therefore, we need to do the search + action under the same > job mutex critical section.

Re: [PATCH v8 09/20] blockjob: rename notifier callbacks as _locked

2022-07-05 Thread Stefan Hajnoczi
On Wed, Jun 29, 2022 at 10:15:27AM -0400, Emanuele Giuseppe Esposito wrote: > They all are called with job_lock held, in job_event_*_locked() > > Signed-off-by: Emanuele Giuseppe Esposito > --- > blockjob.c | 25 +++-- > 1 file changed, 15 insertions(+), 10 deletions(-) Revi

Re: [PATCH v8 05/20] job.c: add job_lock/unlock while keeping job.h intact

2022-07-05 Thread Emanuele Giuseppe Esposito
Am 05/07/2022 um 09:39 schrieb Stefan Hajnoczi: > On Wed, Jun 29, 2022 at 10:15:23AM -0400, Emanuele Giuseppe Esposito wrote: >> +void job_ref(Job *job) >> +{ >> +JOB_LOCK_GUARD(); >> +job_ref_locked(job); >> +} > > You don't need to fix this, but just a note: > > This API seems danger

Re: [PATCH v8 11/20] jobs: use job locks also in the unit tests

2022-07-05 Thread Stefan Hajnoczi
On Wed, Jun 29, 2022 at 10:15:29AM -0400, Emanuele Giuseppe Esposito wrote: > Add missing job synchronization in the unit tests, with > explicit locks. > > We are deliberately using _locked functions wrapped by a guard > instead of a normal call because the normal call will be removed > in future,

Re: [PATCH v8 13/20] jobs: group together API calls under the same job lock

2022-07-05 Thread Stefan Hajnoczi
On Wed, Jun 29, 2022 at 10:15:31AM -0400, Emanuele Giuseppe Esposito wrote: > diff --git a/blockdev.c b/blockdev.c > index 71f793c4ab..5b79093155 100644 > --- a/blockdev.c > +++ b/blockdev.c > @@ -150,12 +150,15 @@ void blockdev_mark_auto_del(BlockBackend *blk) > return; > } > > -

Re: [PATCH v8 08/20] blockjob.h: introduce block_job _locked() APIs

2022-07-05 Thread Emanuele Giuseppe Esposito
Am 05/07/2022 um 09:58 schrieb Stefan Hajnoczi: > On Wed, Jun 29, 2022 at 10:15:26AM -0400, Emanuele Giuseppe Esposito wrote: >> +BlockJob *block_job_next(BlockJob *bjob) >> { >> -Job *job = job_get(id); >> +JOB_LOCK_GUARD(); >> +return block_job_next_locked(bjob); >> +} > > This s

Re: [PATCH v8 13/20] jobs: group together API calls under the same job lock

2022-07-05 Thread Emanuele Giuseppe Esposito
Am 05/07/2022 um 10:14 schrieb Stefan Hajnoczi: > On Wed, Jun 29, 2022 at 10:15:31AM -0400, Emanuele Giuseppe Esposito wrote: >> diff --git a/blockdev.c b/blockdev.c >> index 71f793c4ab..5b79093155 100644 >> --- a/blockdev.c >> +++ b/blockdev.c >> @@ -150,12 +150,15 @@ void blockdev_mark_auto_de

Re: [PATCH 09/18] block: Export blk_pwritev_part() in block-backend-io.h

2022-07-05 Thread Hanna Reitz
On 17.05.22 13:38, Alberto Faria wrote: Also convert it into a generated_co_wrapper. Signed-off-by: Alberto Faria --- block/block-backend.c | 14 -- block/coroutines.h| 5 - include/sysemu/block-backend-io.h | 4 tests/unit/test-block-ioth

Re: [PATCH 08/18] block: Add blk_[co_]preadv_part()

2022-07-05 Thread Hanna Reitz
On 17.05.22 13:38, Alberto Faria wrote: Implement blk_preadv_part() using generated_co_wrapper. Signed-off-by: Alberto Faria --- block/block-backend.c | 30 +++--- block/coroutines.h| 5 - include/sysemu/block-backend-io.h | 7 ++

Re: [PATCH 10/18] block: Change blk_pwrite_compressed() param order

2022-07-05 Thread Hanna Reitz
On 17.05.22 13:38, Alberto Faria wrote: Swap 'buf' and 'bytes' around for consistency with other I/O functions. Signed-off-by: Alberto Faria --- block/block-backend.c | 4 ++-- include/sysemu/block-backend-io.h | 4 ++-- qemu-img.c| 2 +- qemu-io-cmds.c

Re: [PATCH 11/18] block: Add blk_co_pwrite_compressed()

2022-07-05 Thread Hanna Reitz
On 17.05.22 13:38, Alberto Faria wrote: Also convert blk_pwrite_compressed() into a generated_co_wrapper. Signed-off-by: Alberto Faria --- block/block-backend.c | 8 include/sysemu/block-backend-io.h | 7 +-- tests/unit/test-block-iothread.c | 18

Re: [PATCH 12/18] block: Implement blk_pwrite_zeroes() using generated_co_wrapper

2022-07-05 Thread Hanna Reitz
On 17.05.22 13:39, Alberto Faria wrote: Signed-off-by: Alberto Faria --- block/block-backend.c | 8 include/sysemu/block-backend-io.h | 5 +++-- tests/unit/test-block-iothread.c | 17 + 3 files changed, 20 insertions(+), 10 deletions(-) Reviewed-by:

Re: [PATCH 13/18] block: Implement blk_pdiscard() using generated_co_wrapper

2022-07-05 Thread Hanna Reitz
On 17.05.22 13:39, Alberto Faria wrote: Signed-off-by: Alberto Faria --- block/block-backend.c | 12 block/coroutines.h| 3 --- include/sysemu/block-backend-io.h | 3 ++- 3 files changed, 2 insertions(+), 16 deletions(-) Reviewed-by: Hanna Reitz

Re: [PATCH 14/18] block: Implement blk_flush() using generated_co_wrapper

2022-07-05 Thread Hanna Reitz
On 17.05.22 13:39, Alberto Faria wrote: Signed-off-by: Alberto Faria --- block/block-backend.c | 11 --- block/coroutines.h| 2 -- include/sysemu/block-backend-io.h | 2 +- 3 files changed, 1 insertion(+), 14 deletions(-) Reviewed-by: Hanna Reitz

Re: [PATCH 15/18] block: Add blk_co_ioctl()

2022-07-05 Thread Hanna Reitz
On 17.05.22 13:39, Alberto Faria wrote: Also convert blk_ioctl() into a generated_co_wrapper. Signed-off-by: Alberto Faria --- block/block-backend.c | 7 --- block/coroutines.h| 6 -- include/sysemu/block-backend-io.h | 5 - 3 files changed, 8 insert

copy-before-write test failing (was: Re: [PULL 07/10] iotests: copy-before-write: add cases for cbw-timeout option)

2022-07-05 Thread Thomas Huth
On 14/06/2022 12.29, Vladimir Sementsov-Ogievskiy wrote: From: Vladimir Sementsov-Ogievskiy Add two simple test-cases: timeout failure with break-snapshot-on-cbw-error behavior and similar with break-guest-write-on-cbw-error behavior. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: H

Re: [PATCH 16/18] block: Add blk_co_truncate()

2022-07-05 Thread Hanna Reitz
On 17.05.22 13:39, Alberto Faria wrote: Also convert blk_truncate() into a generated_co_wrapper. Signed-off-by: Alberto Faria --- block/block-backend.c | 7 --- include/sysemu/block-backend-io.h | 8 ++-- tests/unit/test-block-iothread.c | 14 ++ 3 files

Re: [PATCH 17/18] block: Reorganize some declarations in block-backend-io.h

2022-07-05 Thread Hanna Reitz
On 17.05.22 13:39, Alberto Faria wrote: Keep generated_co_wrapper and coroutine_fn pairs together. This should make it clear that each I/O function has these two versions. Also move blk_co_{pread,pwrite}()'s implementations out of the header file for consistency. Signed-off-by: Alberto Faria -

Re: [PATCH 18/18] block: Remove remaining unused symbols in coroutines.h

2022-07-05 Thread Hanna Reitz
On 17.05.22 13:39, Alberto Faria wrote: Some can be made static, others are unused generated_co_wrappers. Signed-off-by: Alberto Faria --- block/block-backend.c | 6 +++--- block/coroutines.h| 19 --- 2 files changed, 3 insertions(+), 22 deletions(-) Reviewed-by: Hann

Failure in iotest 183

2022-07-05 Thread Thomas Huth
Hi! I've just hit a failure in iotest 183: --- /home/thuth/devel/qemu/tests/qemu-iotests/183.out +++ /home/thuth/tmp/qemu-build/tests/qemu-iotests/scratch/183/183.out.bad @@ -30,7 +30,7 @@ 'arguments': { 'uri': 'unix:SOCK_DIR/migrate', 'blk': true } } {"return": {}} { 'execute': 'qu

Re: copy-before-write test failing (was: Re: [PULL 07/10] iotests: copy-before-write: add cases for cbw-timeout option)

2022-07-05 Thread Vladimir Sementsov-Ogievskiy
On 7/5/22 12:03, Thomas Huth wrote: On 14/06/2022 12.29, Vladimir Sementsov-Ogievskiy wrote: From: Vladimir Sementsov-Ogievskiy Add two simple test-cases: timeout failure with break-snapshot-on-cbw-error behavior and similar with break-guest-write-on-cbw-error behavior. Signed-off-by: Vladimi

Re: Failure in iotest 183

2022-07-05 Thread Hanna Reitz
On 05.07.22 11:35, Thomas Huth wrote:  Hi! I've just hit a failure in iotest 183: --- /home/thuth/devel/qemu/tests/qemu-iotests/183.out +++ /home/thuth/tmp/qemu-build/tests/qemu-iotests/scratch/183/183.out.bad @@ -30,7 +30,7 @@     'arguments': { 'uri': 'unix:SOCK_DIR/migrate', 'blk': true

Re: Failure in iotest 183

2022-07-05 Thread Thomas Huth
On 05/07/2022 11.58, Hanna Reitz wrote: On 05.07.22 11:35, Thomas Huth wrote:  Hi! I've just hit a failure in iotest 183: --- /home/thuth/devel/qemu/tests/qemu-iotests/183.out +++ /home/thuth/tmp/qemu-build/tests/qemu-iotests/scratch/183/183.out.bad @@ -30,7 +30,7 @@     'arguments': { 'u

Re: [PATCH v8 05/20] job.c: add job_lock/unlock while keeping job.h intact

2022-07-05 Thread Vladimir Sementsov-Ogievskiy
In general looks good to me. On 6/29/22 17:15, Emanuele Giuseppe Esposito wrote: With "intact" we mean that all job.h functions implicitly take the lock. Therefore API callers are unmodified. This means that: - all static functions become _locked, and call _locked functions Some static functi

Re: [PATCH v8 06/20] job.h: define functions called without job lock held

2022-07-05 Thread Vladimir Sementsov-Ogievskiy
On 6/29/22 17:15, Emanuele Giuseppe Esposito wrote: These functions don't need a _locked() counterpart, since they are all called outside job.c and take the lock only internally. Update also the comments in blockjob.c (and move them in job.c). Still, that would be better as a separate patch.

Re: [PATCH v8 06/20] job.h: define functions called without job lock held

2022-07-05 Thread Vladimir Sementsov-Ogievskiy
To subject: hmm, the commit don't define any function.. -- Best regards, Vladimir

Re: [PATCH v8 07/20] job.h: add _locked public functions

2022-07-05 Thread Vladimir Sementsov-Ogievskiy
First, to subject: no function is added in this commit Second, to my comment on previous patch: so, you decided to add "not held" comment to all functions, even that have public _locked() counterpart. Not sure we really need it, but it's OK. Anyway, let's just add all these comments together wi

Re: [RFC 0/8] Introduce an extensible static analyzer

2022-07-05 Thread Alberto Faria
On Tue, Jul 5, 2022 at 8:16 AM Daniel P. Berrangé wrote: > for i in `git ls-tree --name-only -r HEAD:` > do > clang-tidy $i 1>/dev/null 2>&1 > done All of those invocations are probably failing quickly due to missing includes and other problems, since the location of the co

Re: [PATCH v8 16/20] jobs: protect job.aio_context with BQL and job_mutex

2022-07-05 Thread Stefan Hajnoczi
On Wed, Jun 29, 2022 at 10:15:34AM -0400, Emanuele Giuseppe Esposito wrote: > In order to make it thread safe, implement a "fake rwlock", > where we allow reads under BQL *or* job_mutex held, but > writes only under BQL *and* job_mutex. > > The only write we have is in child_job_set_aio_ctx, which

Re: [PATCH v8 13/20] jobs: group together API calls under the same job lock

2022-07-05 Thread Emanuele Giuseppe Esposito
Am 05/07/2022 um 10:17 schrieb Emanuele Giuseppe Esposito: > > > Am 05/07/2022 um 10:14 schrieb Stefan Hajnoczi: >> On Wed, Jun 29, 2022 at 10:15:31AM -0400, Emanuele Giuseppe Esposito wrote: >>> diff --git a/blockdev.c b/blockdev.c >>> index 71f793c4ab..5b79093155 100644 >>> --- a/blockdev.c

Re: [PATCH v8 19/20] blockjob: remove unused functions

2022-07-05 Thread Stefan Hajnoczi
On Wed, Jun 29, 2022 at 10:15:37AM -0400, Emanuele Giuseppe Esposito wrote: > These public functions are not used anywhere, thus can be dropped. > > Signed-off-by: Emanuele Giuseppe Esposito > --- > blockjob.c | 30 -- > include/block/blockjob.h | 33 +++

Re: [PATCH v8 17/20] job.c: enable job lock/unlock and remove Aiocontext locks

2022-07-05 Thread Stefan Hajnoczi
On Wed, Jun 29, 2022 at 10:15:35AM -0400, Emanuele Giuseppe Esposito wrote: > Change the job_{lock/unlock} and macros to use job_mutex. > > Now that they are not nop anymore, remove the aiocontext > to avoid deadlocks. > > Therefore: > - when possible, remove completely the aiocontext lock/unlock

Re: [PATCH v8 00/20] job: replace AioContext lock with job_mutex

2022-07-05 Thread Stefan Hajnoczi
On Wed, Jun 29, 2022 at 10:15:18AM -0400, Emanuele Giuseppe Esposito wrote: > In this series, we want to remove the AioContext lock and instead > use the already existent job_mutex to protect the job structures > and list. This is part of the work to get rid of AioContext lock > usage in favour of

Re: [PATCH v8 20/20] job: remove unused functions

2022-07-05 Thread Stefan Hajnoczi
On Wed, Jun 29, 2022 at 10:15:38AM -0400, Emanuele Giuseppe Esposito wrote: > These public functions are not used anywhere, thus can be dropped. > Also, since this is the final job API that doesn't use AioContext > lock and replaces it with job_lock, adjust all remaining function > documentation to

Re: [PATCH v8 13/20] jobs: group together API calls under the same job lock

2022-07-05 Thread Vladimir Sementsov-Ogievskiy
On 7/5/22 16:01, Emanuele Giuseppe Esposito wrote: Am 05/07/2022 um 10:17 schrieb Emanuele Giuseppe Esposito: Am 05/07/2022 um 10:14 schrieb Stefan Hajnoczi: On Wed, Jun 29, 2022 at 10:15:31AM -0400, Emanuele Giuseppe Esposito wrote: diff --git a/blockdev.c b/blockdev.c index 71f793c4ab..5

Re: [PATCH v2] io_uring: fix short read slow path

2022-07-05 Thread Stefan Hajnoczi
On Fri, Jul 01, 2022 at 07:52:31AM +0900, Dominique Martinet wrote: > Stefano Garzarella wrote on Thu, Jun 30, 2022 at 05:49:21PM +0200: > > > so when we ask for more we issue an extra short reads, making sure we go > > > through the two short reads path. > > > (Unfortunately I wasn't quite sure wh

Re: [PATCH v2] io_uring: fix short read slow path

2022-07-05 Thread Stefan Hajnoczi
On Thu, Jun 30, 2022 at 10:01:37AM +0900, Dominique Martinet wrote: > sqeq.off here is the offset to read within the disk image, so obviously > not 'nread' (the amount we just read), but as the author meant to write > its current value incremented by the amount we just read. > > Normally recent ve

Re: [PATCH 1/8] virtio_queue_aio_attach_host_notifier: remove AioContext lock

2022-07-05 Thread Stefan Hajnoczi
On Thu, Jun 09, 2022 at 10:37:20AM -0400, Emanuele Giuseppe Esposito wrote: > @@ -146,7 +147,6 @@ int virtio_scsi_dataplane_start(VirtIODevice *vdev) > > s->dataplane_starting = false; > s->dataplane_started = true; > -aio_context_release(s->ctx); > return 0; This looks risky

Re: [PATCH 2/8] block-backend: enable_write_cache should be atomic

2022-07-05 Thread Stefan Hajnoczi
On Thu, Jun 09, 2022 at 10:37:21AM -0400, Emanuele Giuseppe Esposito wrote: > It is read from IO_CODE and written with BQL held, > so setting it as atomic should be enough. > > Also remove the aiocontext lock that was sporadically > taken around the set. > > Signed-off-by: Emanuele Giuseppe Espos

Re: [PATCH 3/8] virtio_blk_process_queued_requests: always run in a bh

2022-07-05 Thread Stefan Hajnoczi
On Thu, Jun 09, 2022 at 10:37:22AM -0400, Emanuele Giuseppe Esposito wrote: > diff --git a/hw/block/dataplane/virtio-blk.c b/hw/block/dataplane/virtio-blk.c > index f9224f23d2..03e10a36a4 100644 > --- a/hw/block/dataplane/virtio-blk.c > +++ b/hw/block/dataplane/virtio-blk.c > @@ -234,8 +234,16 @@ i

Re: [PATCH 4/8] virtio: categorize callbacks in GS

2022-07-05 Thread Stefan Hajnoczi
On Thu, Jun 09, 2022 at 10:37:23AM -0400, Emanuele Giuseppe Esposito wrote: > All the callbacks below are always running in the main loop. > > The callbacks are the following: > - start/stop_ioeventfd: these are the callbacks where > blk_set_aio_context(iothread) is done, so they are called in t

Re: [PATCH 5/8] virtio-blk: mark GLOBAL_STATE_CODE functions

2022-07-05 Thread Stefan Hajnoczi
On Thu, Jun 09, 2022 at 10:37:24AM -0400, Emanuele Giuseppe Esposito wrote: > Just as done in the block API, mark functions in virtio-blk > that are always called in the main loop with BQL held. > > We know such functions are GS because they all are callbacks > from virtio.c API that has already c

Re: [PATCH 6/8] virtio-blk: mark IO_CODE functions

2022-07-05 Thread Stefan Hajnoczi
On Thu, Jun 09, 2022 at 10:37:25AM -0400, Emanuele Giuseppe Esposito wrote: > Just as done in the block API, mark functions in virtio-blk > that are called also from iothread(s). > > We know such functions are IO because many are blk_* callbacks, > running always in the device iothread, and remain

Re: [PATCH 7/8] VirtIOBlock: protect rq with its own lock

2022-07-05 Thread Stefan Hajnoczi
On Thu, Jun 09, 2022 at 10:37:26AM -0400, Emanuele Giuseppe Esposito wrote: > @@ -946,17 +955,20 @@ static void virtio_blk_reset(VirtIODevice *vdev) > * stops all Iothreads. > */ > blk_drain(s->blk); > +aio_context_release(ctx); > > /* We drop queued requests after blk_d

Re: [PATCH 8/8] virtio-blk: remove unnecessary AioContext lock from function already safe

2022-07-05 Thread Stefan Hajnoczi
On Thu, Jun 09, 2022 at 10:37:27AM -0400, Emanuele Giuseppe Esposito wrote: > AioContext lock was introduced in b9e413dd375 and in this instance > it is used to protect these 3 functions: > - virtio_blk_handle_rw_error > - virtio_blk_req_complete > - block_acct_done > > Now that all three of the a

Re: [PATCH v8 13/20] jobs: group together API calls under the same job lock

2022-07-05 Thread Vladimir Sementsov-Ogievskiy
On 6/29/22 17:15, Emanuele Giuseppe Esposito wrote: --- a/job.c +++ b/job.c @@ -1045,11 +1045,14 @@ static void job_completed_txn_abort_locked(Job *job) /* Called with job_mutex held, but releases it temporarily */ static int job_prepare_locked(Job *job) { +int ret; + GLOBAL_STATE

Re: [PATCH v8 08/20] blockjob.h: introduce block_job _locked() APIs

2022-07-05 Thread Vladimir Sementsov-Ogievskiy
On 6/29/22 17:15, Emanuele Giuseppe Esposito wrote: Just as done with job.h, create _locked() functions in blockjob.h We modify not only blockjob.h, I'd s/blockjob.h/blockjob/ in subject. Also, we start to introduce _locked block_job_* APIs. Does it mean that BlockJob and Job share the global

Re: [PATCH 17/18] block: Reorganize some declarations in block-backend-io.h

2022-07-05 Thread Alberto Faria
On Tue, Jul 5, 2022 at 10:18 AM Hanna Reitz wrote: > This moves blk_co_copy_range() from the “I/O API functions” section of > this header into the “"I/O or GS" API functions” section. Is that intended? Oops, thanks, it wasn't intended. Will fix. Alberto

[PATCH] iotests: fix copy-before-write for macOS and FreeBSD

2022-07-05 Thread Vladimir Sementsov-Ogievskiy
strerror() represents ETIMEDOUT a bit different in Linux and macOS / FreeBSD. Let's support the latter too. Fixes: 9d05a87b77 ("iotests: copy-before-write: add cases for cbw-timeout option") Signed-off-by: Vladimir Sementsov-Ogievskiy --- As John and Thomas noted, the new iotests fails for Free

Re: [RFC 0/8] Introduce an extensible static analyzer

2022-07-05 Thread Daniel P . Berrangé
On Tue, Jul 05, 2022 at 12:28:55PM +0100, Alberto Faria wrote: > On Tue, Jul 5, 2022 at 8:16 AM Daniel P. Berrangé wrote: > > for i in `git ls-tree --name-only -r HEAD:` > > do > > clang-tidy $i 1>/dev/null 2>&1 > > done > > All of those invocations are probably failing qui

Re: [RFC 0/8] Introduce an extensible static analyzer

2022-07-05 Thread Daniel P . Berrangé
On Sat, Jul 02, 2022 at 12:33:23PM +0100, Alberto Faria wrote: > This series introduces a static analyzer for QEMU. It consists of a > single static-analyzer.py script that relies on libclang's Python > bindings, and provides a common framework on which arbitrary static > analysis checks can be dev

[PATCH v2 00/18] Make block-backend-io.h API more consistent

2022-07-05 Thread Alberto Faria
Adjust existing pairs of non-coroutine and coroutine functions to share the same calling convention, and add non-coroutine/coroutine counterparts where they don't exist. Also make the non-coroutine versions generated_co_wrappers. This series sits on top of "[PATCH v5 00/10] Implement bdrv_{pread,

[PATCH v2 01/18] block: Make blk_{pread, pwrite}() return 0 on success

2022-07-05 Thread Alberto Faria
They currently return the value of their 'bytes' parameter on success. Make them return 0 instead, for consistency with other I/O functions and in preparation to implement them using generated_co_wrapper. This also makes it clear that short reads/writes are not possible. Signed-off-by: Alberto Fa

[PATCH v2 02/18] block: Add a 'flags' param to blk_pread()

2022-07-05 Thread Alberto Faria
For consistency with other I/O functions, and in preparation to implement it using generated_co_wrapper. Callers were updated using this Coccinelle script: @@ expression blk, offset, buf, bytes; @@ - blk_pread(blk, offset, buf, bytes) + blk_pread(blk, offset, buf, bytes, 0) It had no

[PATCH v2 03/18] block: Change blk_{pread,pwrite}() param order

2022-07-05 Thread Alberto Faria
Swap 'buf' and 'bytes' around for consistency with blk_co_{pread,pwrite}(), and in preparation to implement these functions using generated_co_wrapper. Callers were updated using this Coccinelle script: @@ expression blk, offset, buf, bytes, flags; @@ - blk_pread(blk, offset, buf, bytes,

[PATCH v2 04/18] block: Make 'bytes' param of blk_{pread, pwrite}() an int64_t

2022-07-05 Thread Alberto Faria
For consistency with other I/O functions, and in preparation to implement them using generated_co_wrapper. Signed-off-by: Alberto Faria Reviewed-by: Paolo Bonzini Reviewed-by: Hanna Reitz --- block/block-backend.c | 6 +++--- include/sysemu/block-backend-io.h | 6 +++--- 2 files ch

[PATCH v2 05/18] block: Make blk_co_pwrite() take a const buffer

2022-07-05 Thread Alberto Faria
It does not mutate the buffer. Signed-off-by: Alberto Faria Reviewed-by: Paolo Bonzini Reviewed-by: Hanna Reitz --- include/sysemu/block-backend-io.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/sysemu/block-backend-io.h b/include/sysemu/block-backend-io.h index

[PATCH v2 06/18] block: Implement blk_{pread, pwrite}() using generated_co_wrapper

2022-07-05 Thread Alberto Faria
We need to add include/sysemu/block-backend-io.h to the inputs of the block-gen.c target defined in block/meson.build. Signed-off-by: Alberto Faria Reviewed-by: Hanna Reitz --- block/block-backend.c | 23 --- block/coroutines.h| 4 block/mes

[PATCH v2 09/18] block: Export blk_pwritev_part() in block-backend-io.h

2022-07-05 Thread Alberto Faria
Also convert it into a generated_co_wrapper. Signed-off-by: Alberto Faria Reviewed-by: Paolo Bonzini Reviewed-by: Hanna Reitz --- block/block-backend.c | 14 -- block/coroutines.h| 5 - include/sysemu/block-backend-io.h | 4 tests/unit/test-bl

[PATCH v2 07/18] block: Add blk_{preadv,pwritev}()

2022-07-05 Thread Alberto Faria
Implement them using generated_co_wrapper. Signed-off-by: Alberto Faria Reviewed-by: Paolo Bonzini Reviewed-by: Hanna Reitz --- include/sysemu/block-backend-io.h | 6 + tests/unit/test-block-iothread.c | 42 ++- 2 files changed, 47 insertions(+), 1 deletion(-)

[PATCH v2 10/18] block: Change blk_pwrite_compressed() param order

2022-07-05 Thread Alberto Faria
Swap 'buf' and 'bytes' around for consistency with other I/O functions. Signed-off-by: Alberto Faria Reviewed-by: Paolo Bonzini Reviewed-by: Hanna Reitz --- block/block-backend.c | 4 ++-- include/sysemu/block-backend-io.h | 4 ++-- qemu-img.c| 2 +- qemu-io

[PATCH v2 11/18] block: Add blk_co_pwrite_compressed()

2022-07-05 Thread Alberto Faria
Also convert blk_pwrite_compressed() into a generated_co_wrapper. Signed-off-by: Alberto Faria Reviewed-by: Paolo Bonzini Reviewed-by: Hanna Reitz --- block/block-backend.c | 8 include/sysemu/block-backend-io.h | 7 +-- tests/unit/test-block-iothread.c | 18 +++

[PATCH v2 08/18] block: Add blk_[co_]preadv_part()

2022-07-05 Thread Alberto Faria
Implement blk_preadv_part() using generated_co_wrapper. Signed-off-by: Alberto Faria Reviewed-by: Paolo Bonzini Reviewed-by: Hanna Reitz --- block/block-backend.c | 30 +++--- block/coroutines.h| 5 - include/sysemu/block-backend-io.h |

[PATCH v2 12/18] block: Implement blk_pwrite_zeroes() using generated_co_wrapper

2022-07-05 Thread Alberto Faria
Signed-off-by: Alberto Faria Reviewed-by: Paolo Bonzini Reviewed-by: Hanna Reitz --- block/block-backend.c | 8 include/sysemu/block-backend-io.h | 5 +++-- tests/unit/test-block-iothread.c | 17 + 3 files changed, 20 insertions(+), 10 deletions(-) diff

[PATCH v2 14/18] block: Implement blk_flush() using generated_co_wrapper

2022-07-05 Thread Alberto Faria
Signed-off-by: Alberto Faria Reviewed-by: Paolo Bonzini Reviewed-by: Hanna Reitz --- block/block-backend.c | 11 --- block/coroutines.h| 2 -- include/sysemu/block-backend-io.h | 2 +- 3 files changed, 1 insertion(+), 14 deletions(-) diff --git a/block/blo

[PATCH v2 17/18] block: Reorganize some declarations in block-backend-io.h

2022-07-05 Thread Alberto Faria
Keep generated_co_wrapper and coroutine_fn pairs together. This should make it clear that each I/O function has these two versions. Also move blk_co_{pread,pwrite}()'s implementations out of the header file for consistency. Signed-off-by: Alberto Faria Reviewed-by: Paolo Bonzini --- block/bloc

[PATCH v2 15/18] block: Add blk_co_ioctl()

2022-07-05 Thread Alberto Faria
Also convert blk_ioctl() into a generated_co_wrapper. Signed-off-by: Alberto Faria Reviewed-by: Paolo Bonzini Reviewed-by: Hanna Reitz --- block/block-backend.c | 7 --- block/coroutines.h| 6 -- include/sysemu/block-backend-io.h | 5 - 3 files changed,

[PATCH v2 13/18] block: Implement blk_pdiscard() using generated_co_wrapper

2022-07-05 Thread Alberto Faria
Signed-off-by: Alberto Faria Reviewed-by: Paolo Bonzini Reviewed-by: Hanna Reitz --- block/block-backend.c | 12 block/coroutines.h| 3 --- include/sysemu/block-backend-io.h | 3 ++- 3 files changed, 2 insertions(+), 16 deletions(-) diff --git a/block

[PATCH v2 16/18] block: Add blk_co_truncate()

2022-07-05 Thread Alberto Faria
Also convert blk_truncate() into a generated_co_wrapper. Signed-off-by: Alberto Faria Reviewed-by: Paolo Bonzini Reviewed-by: Hanna Reitz --- block/block-backend.c | 7 --- include/sysemu/block-backend-io.h | 8 ++-- tests/unit/test-block-iothread.c | 14 ++

[PATCH v2 18/18] block: Remove remaining unused symbols in coroutines.h

2022-07-05 Thread Alberto Faria
Some can be made static, others are unused generated_co_wrappers. Signed-off-by: Alberto Faria Reviewed-by: Paolo Bonzini Reviewed-by: Hanna Reitz --- block/block-backend.c | 6 +++--- block/coroutines.h| 19 --- 2 files changed, 3 insertions(+), 22 deletions(-) diff --gi

Re: [PATCH] iotests: fix copy-before-write for macOS and FreeBSD

2022-07-05 Thread Richard Henderson
On 7/5/22 21:07, Vladimir Sementsov-Ogievskiy wrote: strerror() represents ETIMEDOUT a bit different in Linux and macOS / FreeBSD. Let's support the latter too. Fixes: 9d05a87b77 ("iotests: copy-before-write: add cases for cbw-timeout option") Signed-off-by: Vladimir Sementsov-Ogievskiy --- A

Re: [PATCH] iotests: fix copy-before-write for macOS and FreeBSD

2022-07-05 Thread Vladimir Sementsov-Ogievskiy
On 7/5/22 20:22, Richard Henderson wrote: On 7/5/22 21:07, Vladimir Sementsov-Ogievskiy wrote: strerror() represents ETIMEDOUT a bit different in Linux and macOS / FreeBSD. Let's support the latter too. Fixes: 9d05a87b77 ("iotests: copy-before-write: add cases for cbw-timeout option") Signed-o

Re: [PATCH v2] io_uring: fix short read slow path

2022-07-05 Thread Jens Axboe
On 7/5/22 7:28 AM, Stefan Hajnoczi wrote: > On Fri, Jul 01, 2022 at 07:52:31AM +0900, Dominique Martinet wrote: >> Stefano Garzarella wrote on Thu, Jun 30, 2022 at 05:49:21PM +0200: so when we ask for more we issue an extra short reads, making sure we go through the two short reads path.

Re: [PATCH v2] io_uring: fix short read slow path

2022-07-05 Thread Dominique Martinet
Stefan Hajnoczi wrote on Tue, Jul 05, 2022 at 02:28:08PM +0100: > > The older kernel I have installed right now is 5.16 and that can > > reproduce it -- I'll give my laptop some work over the weekend to test > > still maintained stable branches if that's useful. > > Linux 5.16 contains commit 9d9