Re: [PATCH v7 10/18] jobs: rename static functions called with job_mutex held

2022-06-24 Thread Kevin Wolf
Am 23.06.2022 um 13:19 hat Emanuele Giuseppe Esposito geschrieben: > > > Am 23/06/2022 um 13:10 schrieb Vladimir Sementsov-Ogievskiy: > > On 6/23/22 12:08, Emanuele Giuseppe Esposito wrote: > >> > >> > >> Am 22/06/2022 um 20:38 schrieb Vladimir Sementsov-Ogievskiy: > >>> On 6/22/22 17:26, Emanuel

Re: [PATCH v7 10/18] jobs: rename static functions called with job_mutex held

2022-06-24 Thread Paolo Bonzini
On 6/24/22 16:29, Kevin Wolf wrote: Yes, I think Vladimir is having the same difficulties with reading the series as I had. And I believe his suggestion would make the intermediate states less impossible to review. The question is how much work it would be and whether you're willing to do this. A

[PULL v2 00/20] Block layer patches

2022-06-24 Thread Kevin Wolf
The following changes since commit 3a821c52e1a30ecd9a436f2c67cc66b5628c829f: Merge tag 'nvme-next-pull-request' of git://git.infradead.org/qemu-nvme into staging (2022-06-23 14:52:30 -0700) are available in the Git repository at: git://repo.or.cz/qemu/kevin.git tags/for-upstream for you to

[PULL v2 06/20] block: Support passing NULL ops to blk_set_dev_ops()

2022-06-24 Thread Kevin Wolf
From: Xie Yongji This supports passing NULL ops to blk_set_dev_ops() so that we can remove stale ops in some cases. Signed-off-by: Xie Yongji Reviewed-by: Stefan Hajnoczi Message-Id: <20220523084611.91-2-xieyon...@bytedance.com> Signed-off-by: Kevin Wolf --- block/block-backend.c | 2 +- 1 f

[PULL v2 11/20] vduse-blk: Implement vduse-blk export

2022-06-24 Thread Kevin Wolf
From: Xie Yongji This implements a VDUSE block backends based on the libvduse library. We can use it to export the BDSs for both VM and container (host) usage. The new command-line syntax is: $ qemu-storage-daemon \ --blockdev file,node-name=drive0,filename=test.img \ --export vduse-blk

[PULL v2 03/20] block: block_dirty_bitmap_merge(): fix error path

2022-06-24 Thread Kevin Wolf
From: Vladimir Sementsov-Ogievskiy At the end we ignore failure of bdrv_merge_dirty_bitmap() and report success. And still set errp. That's wrong. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Nikita Lapshin Reviewed-by: Kevin Wolf Message-Id: <20220517111206.23585-2-v.sementsov...

[PULL v2 01/20] block: drop unused bdrv_co_drain() API

2022-06-24 Thread Kevin Wolf
From: Stefan Hajnoczi bdrv_co_drain() has not been used since commit 9a0cec664eef ("mirror: use bdrv_drained_begin/bdrv_drained_end") in 2016. Remove it so there are fewer drain scenarios to worry about. Use bdrv_drained_begin()/bdrv_drained_end() instead. They are "mixed" functions that can be

[PULL v2 04/20] block: improve block_dirty_bitmap_merge(): don't allocate extra bitmap

2022-06-24 Thread Kevin Wolf
From: Vladimir Sementsov-Ogievskiy We don't need extra bitmap. All we need is to backup the original bitmap when we do first merge. So, drop extra temporary bitmap and work directly with target and backup. Still to keep old semantics, that on failure target is unchanged and user don't need to re

[PULL v2 13/20] libvduse: Add support for reconnecting

2022-06-24 Thread Kevin Wolf
From: Xie Yongji To support reconnecting after restart or crash, VDUSE backend might need to resubmit inflight I/Os. This stores the metadata such as the index of inflight I/O's descriptors to a shm file so that VDUSE backend can restore them during reconnecting. Signed-off-by: Xie Yongji Messa

[PULL v2 09/20] linux-headers: Add vduse.h

2022-06-24 Thread Kevin Wolf
From: Xie Yongji This adds vduse header to linux headers so that the relevant VDUSE API can be used in subsequent patches. Signed-off-by: Xie Yongji Reviewed-by: Stefan Hajnoczi Message-Id: <20220523084611.91-5-xieyon...@bytedance.com> Signed-off-by: Kevin Wolf --- linux-headers/linux/vduse.

[PULL v2 08/20] block/export: Abstract out the logic of virtio-blk I/O process

2022-06-24 Thread Kevin Wolf
From: Xie Yongji Abstract the common logic of virtio-blk I/O process to a function named virtio_blk_process_req(). It's needed for the following commit. Signed-off-by: Xie Yongji Message-Id: <20220523084611.91-4-xieyon...@bytedance.com> Reviewed-by: Stefan Hajnoczi Signed-off-by: Kevin Wolf -

[PULL v2 02/20] block: get rid of blk->guest_block_size

2022-06-24 Thread Kevin Wolf
From: Stefan Hajnoczi Commit 1b7fd729559c ("block: rename buffer_alignment to guest_block_size") noted: At this point, the field is set by the device emulation, but completely ignored by the block layer. The last time the value of buffer_alignment/guest_block_size was actually used was befo

[PULL v2 05/20] block: simplify handling of try to merge different sized bitmaps

2022-06-24 Thread Kevin Wolf
From: Vladimir Sementsov-Ogievskiy We have too much logic to simply check that bitmaps are of the same size. Let's just define that hbitmap_merge() and bdrv_dirty_bitmap_merge_internal() require their argument bitmaps be of same size, this simplifies things. Let's look through the callers: For

[PULL v2 07/20] block/export: Fix incorrect length passed to vu_queue_push()

2022-06-24 Thread Kevin Wolf
From: Xie Yongji Now the req->size is set to the correct value only when handling VIRTIO_BLK_T_GET_ID request. This patch fixes it. Signed-off-by: Xie Yongji Message-Id: <20220523084611.91-3-xieyon...@bytedance.com> Reviewed-by: Stefan Hajnoczi Signed-off-by: Kevin Wolf --- block/export/vhos

[PULL v2 10/20] libvduse: Add VDUSE (vDPA Device in Userspace) library

2022-06-24 Thread Kevin Wolf
From: Xie Yongji VDUSE [1] is a linux framework that makes it possible to implement software-emulated vDPA devices in userspace. This adds a library as a subproject to help implementing VDUSE backends in QEMU. [1] https://www.kernel.org/doc/html/latest/userspace-api/vduse.html Signed-off-by: Xi

[PULL v2 17/20] aio_wait_kick: add missing memory barrier

2022-06-24 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito It seems that aio_wait_kick always required a memory barrier or atomic operation in the caller, but nobody actually took care of doing it. Let's put the barrier in the function instead, and pair it with another one in AIO_WAIT_WHILE. Read aio_wait_kick() comment

[PULL v2 18/20] nbd: Drop dead code spotted by Coverity

2022-06-24 Thread Kevin Wolf
From: Eric Blake CID 1488362 points out that the second 'rc >= 0' check is now dead code. Reported-by: Peter Maydell Fixes: 172f5f1a40(nbd: remove peppering of nbd_client_connected) Signed-off-by: Eric Blake Message-Id: <20220516210519.76135-1-ebl...@redhat.com> Reviewed-by: Peter Maydell Rev

[PULL v2 20/20] vduse-blk: Add name option

2022-06-24 Thread Kevin Wolf
From: Xie Yongji Currently we use 'id' option as the name of VDUSE device. It's a bit confusing since we use one value for two different purposes: the ID to identfy the export within QEMU (must be distinct from any other exports in the same QEMU process, but can overlap with names used by other p

[PULL v2 12/20] vduse-blk: Add vduse-blk resize support

2022-06-24 Thread Kevin Wolf
From: Xie Yongji To support block resize, this uses vduse_dev_update_config() to update the capacity field in configuration space and inject config interrupt on the block resize callback. Signed-off-by: Xie Yongji Reviewed-by: Stefan Hajnoczi Message-Id: <20220523084611.91-8-xieyon...@bytedanc

[PULL v2 14/20] qsd: document vduse-blk exports

2022-06-24 Thread Kevin Wolf
From: Stefan Hajnoczi Document vduse-blk exports in qemu-storage-daemon --help and the qemu-storage-daemon(1) man page. Based-on: <20220523084611.91-1-xieyon...@bytedance.com> Cc: Xie Yongji Signed-off-by: Stefan Hajnoczi Message-Id: <20220525121947.859820-1-stefa...@redhat.com> Signed-off-by:

[PULL v2 19/20] vduse-blk: Add serial option

2022-06-24 Thread Kevin Wolf
From: Xie Yongji Add a 'serial' option to allow user to specify this value explicitly. And the default value is changed to an empty string as what we did in "hw/block/virtio-blk.c". Signed-off-by: Xie Yongji Message-Id: <20220614051532.92-6-xieyon...@bytedance.com> Signed-off-by: Kevin Wolf --

Re: [RFC v2] Adding block layer APIs resembling Linux ZoneBlockDevice ioctls.

2022-06-24 Thread Stefan Hajnoczi
On Fri, Jun 24, 2022 at 11:14:32AM +0800, Sam Li wrote: > Hi Stefan, > > Stefan Hajnoczi 于2022年6月20日周一 15:55写道: > > > > On Mon, Jun 20, 2022 at 11:36:11AM +0800, Sam Li wrote: > > > > Hi Sam, > > Is this version 2 of "[RFC v1] Add support for zoned device"? Please > > keep the email subject line

[PULL v2 16/20] block/gluster: correctly set max_pdiscard

2022-06-24 Thread Kevin Wolf
From: Fabian Ebner On 64-bit platforms, assigning SIZE_MAX to the int64_t max_pdiscard results in a negative value, and the following assertion would trigger down the line (it's not the same max_pdiscard, but computed from the other one): qemu-system-x86_64: ../block/io.c:3166: bdrv_co_pdiscard:

[PULL v2 15/20] block/rbd: report a better error when namespace does not exist

2022-06-24 Thread Kevin Wolf
From: Stefano Garzarella If the namespace does not exist, rbd_create() fails with -ENOENT and QEMU reports a generic "error rbd create: No such file or directory": $ qemu-img create rbd:rbd/namespace/image 1M Formatting 'rbd:rbd/namespace/image', fmt=raw size=1048576 qemu-img: rbd:rb

Re: [RFC v2] Adding block layer APIs resembling Linux ZoneBlockDevice ioctls.

2022-06-24 Thread Sam Li
Stefan Hajnoczi 于2022年6月24日周五 23:50写道: > > On Fri, Jun 24, 2022 at 11:14:32AM +0800, Sam Li wrote: > > Hi Stefan, > > > > Stefan Hajnoczi 于2022年6月20日周一 15:55写道: > > > > > > On Mon, Jun 20, 2022 at 11:36:11AM +0800, Sam Li wrote: > > > > > > Hi Sam, > > > Is this version 2 of "[RFC v1] Add support

Re: [PATCH v7 10/18] jobs: rename static functions called with job_mutex held

2022-06-24 Thread Emanuele Giuseppe Esposito
Am 24/06/2022 um 17:28 schrieb Paolo Bonzini: > On 6/24/22 16:29, Kevin Wolf wrote: >> Yes, I think Vladimir is having the same difficulties with reading the >> series as I had. And I believe his suggestion would make the >> intermediate states less impossible to review. The question is how much

Re: [PATCH v7 01/18] job.c: make job_mutex and job_lock/unlock() public

2022-06-24 Thread Vladimir Sementsov-Ogievskiy
I've already acked this (honestly, because Stefan do), but still, want to clarify: On 6/16/22 16:18, Emanuele Giuseppe Esposito wrote: job mutex will be used to protect the job struct elements and list, replacing AioContext locks. Right now use a shared lock for all jobs, in order to keep thin

[PATCH v4] hw: m25p80: add tests for write protect (WP# and SRWD bit)

2022-06-24 Thread Iris Chen
Signed-off-by: Iris Chen --- Adding Signed Off By tag -- sorry I missed that ! tests/qtest/aspeed_smc-test.c | 62 +++ 1 file changed, 62 insertions(+) diff --git a/tests/qtest/aspeed_smc-test.c b/tests/qtest/aspeed_smc-test.c index ec233315e6..7786addfb8 100644

[PATCH] python: QEMUMachine: enable qmp accept timeout by default

2022-06-24 Thread Vladimir Sementsov-Ogievskiy
I've spent much time trying to debug hanging pipeline in gitlab. I started from and idea that I have problem in code in my series (which has some timeouts). Finally I found that the problem is that I've used QEMUMachine class directly to avoid qtest, and didn't add necessary arguments. Qemu fails a

Re: [PATCH v3] spec: Clarify BLOCK_STATUS reply details

2022-06-24 Thread Eric Blake
On Mon, Apr 04, 2022 at 11:48:46AM +0100, Richard W.M. Jones wrote: Whoops, looks like I let this one sit for a while. > On Fri, Apr 01, 2022 at 04:08:07PM -0500, Eric Blake wrote: > > Our docs were inconsistent on whether a NBD_REPLY_TYPE_BLOCK_STATUS > > reply chunk can exceed the client's requ

Re: [PULL v2 00/20] Block layer patches

2022-06-24 Thread Richard Henderson
On 6/24/22 08:40, Kevin Wolf wrote: The following changes since commit 3a821c52e1a30ecd9a436f2c67cc66b5628c829f: Merge tag 'nvme-next-pull-request' of git://git.infradead.org/qemu-nvme into staging (2022-06-23 14:52:30 -0700) are available in the Git repository at: git://repo.or.cz/qemu

[PATCH v6 00/15] block: cleanup backing and file handling

2022-06-24 Thread Vladimir Sementsov-Ogievskiy
Hi all! That's the first part of "[PATCH v5 00/45] Transactional block-graph modifying API", updated and almost reviewed. On commit (15) is added to original scope of "block: cleanup backing and file handling", as it's related. 01: add Hanna's r-b 02: - mention snapshot-access in commit msg

[PATCH v6 03/15] block/blklogwrites: don't care to remove bs->file child on failure

2022-06-24 Thread Vladimir Sementsov-Ogievskiy
We don't need to remove bs->file, generic layer takes care of it. No other driver cares to remove bs->file on failure by hand. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Hanna Reitz --- block/blklogwrites.c | 4 1 file changed, 4 deletions(-) diff --git a/block/blklogwrites.

[PATCH v6 01/15] block: BlockDriver: add .filtered_child_is_backing field

2022-06-24 Thread Vladimir Sementsov-Ogievskiy
Unfortunately not all filters use .file child as filtered child. Two exclusions are mirror_top and commit_top. Happily they both are private filters. Bad thing is that this inconsistency is observable through qmp commands query-block / query-named-block-nodes. So, could we just change mirror_top an

[PATCH v6 05/15] tests-bdrv-drain: bdrv_replace_test driver: declare supports_backing

2022-06-24 Thread Vladimir Sementsov-Ogievskiy
We do add COW child to the node. In future we are going to forbid adding COW child to the node that doesn't support backing. So, fix it here now. Don't worry about setting bs->backing itself: in further commit we'll update the block-layer to automatically set/unset this field in generic code. Si

[PATCH v6 06/15] test-bdrv-graph-mod: fix filters to be filters

2022-06-24 Thread Vladimir Sementsov-Ogievskiy
bdrv_pass_through is used as filter, even all node variables has corresponding names. We want to append it, so it should be backing-child-based filter like mirror_top. So, in test_update_perm_tree, first child should be DATA, as we don't want filters with two filtered children. bdrv_exclusive_writ

[PATCH v6 04/15] test-bdrv-graph-mod: update test_parallel_perm_update test case

2022-06-24 Thread Vladimir Sementsov-Ogievskiy
test_parallel_perm_update() does two things that we are going to restrict in the near future: 1. It updates bs->file field by hand. bs->file will be managed automatically by generic code (together with bs->children list). Let's better refactor our "tricky" bds to have own state where one

[PATCH v6 08/15] block/snapshot: stress that we fallback to primary child

2022-06-24 Thread Vladimir Sementsov-Ogievskiy
Actually what we chose is a primary child. Let's stress it in the code. We are going to drop indirect pointer logic here in future. Actually this commit simplifies the future work: we drop use of indirection in the assertion now. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Hanna Rei

[PATCH v6 02/15] block: introduce bdrv_open_file_child() helper

2022-06-24 Thread Vladimir Sementsov-Ogievskiy
Almost all drivers call bdrv_open_child() similarly. Let's create a helper for this. The only not updated drivers that call bdrv_open_child() to set bs->file are raw-format and snapshot-access: raw-format sometimes want to have filtered child but don't set drv->is_filter to true. s

[PATCH v6 09/15] Revert "block: Let replace_child_noperm free children"

2022-06-24 Thread Vladimir Sementsov-Ogievskiy
We are going to reimplement this behavior (clear bs->file / bs->backing pointers automatically when child->bs is cleared) in a nicer way, see further commit "block: Manipulate bs->file / bs->backing pointers in .attach/.detach". With this revert we bring back a problem that was fixed by b0a9f6fed3

[PATCH v6 15/15] block: refactor bdrv_remove_file_or_backing_child to bdrv_remove_child

2022-06-24 Thread Vladimir Sementsov-Ogievskiy
Now the function can remove any child, so give it more common name. Drop assertions and drop bs argument which becomes unused. Function would be reused in a further commit. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 27 +-- 1 file changed, 9 insertions(+),

[PATCH v6 07/15] block: document connection between child roles and bs->backing/bs->file

2022-06-24 Thread Vladimir Sementsov-Ogievskiy
Make the informal rules formal. In further commit we'll add corresponding assertions. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block-common.h | 39 1 file changed, 39 insertions(+) diff --git a/include/block/block-common.h b/include/bloc

[PATCH v6 13/15] block: Manipulate bs->file / bs->backing pointers in .attach/.detach

2022-06-24 Thread Vladimir Sementsov-Ogievskiy
bs->file and bs->backing are a kind of duplication of part of bs->children. But very useful diplication, so let's not drop them at all:) We should manage bs->file and bs->backing in same place, where we manage bs->children, to keep them in sync. Moreover, generic io paths are unprepared to BdrvCh

[PATCH v6 14/15] block/snapshot: drop indirection around bdrv_snapshot_fallback_ptr

2022-06-24 Thread Vladimir Sementsov-Ogievskiy
Now the indirection is not actually used, we can safely reduce it to simple pointer. For consistency do a bit of refactoring to get rid of _ptr suffixes that become meaningless. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/snapshot.c | 38 -- 1 file c

[PATCH v6 10/15] Revert "block: Let replace_child_tran keep indirect pointer"

2022-06-24 Thread Vladimir Sementsov-Ogievskiy
That's a preparation to previously reverted "block: Let replace_child_noperm free children". Drop it too, we don't need it for a new approach. This reverts commit 82b54cf51656bf3cd5ed1ac549e8a1085a0e3290. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 81 +++--

[PATCH v6 12/15] Revert "block: Pass BdrvChild ** to replace_child_noperm"

2022-06-24 Thread Vladimir Sementsov-Ogievskiy
That's a preparation to previously reverted "block: Let replace_child_noperm free children". Drop it too, we don't need it for a new approach. This reverts commit be64bbb0149748f3999c49b13976aafb8330ea86. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 23 +++ 1 fi

[PATCH v6 11/15] Revert "block: Restructure remove_file_or_backing_child()"

2022-06-24 Thread Vladimir Sementsov-Ogievskiy
That's a preparation to previously reverted "block: Let replace_child_noperm free children". Drop it too, we don't need it for a new approach. This reverts commit 562bda8bb41879eeda0bd484dd3d55134579b28e. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 21 + 1 file