Hi, This series: - Fixes some bugs/minor inconveniences, - Removes libfuse from the request processing path, - Make the FUSE export use coroutines for request handling, - Introduces multi-threading into the FUSE export.
More detail on the v1 cover letter: https://lists.nongnu.org/archive/html/qemu-block/2025-03/msg00359.html Changes from v1: - Patch 1: Clarified “polling” to be `aio_poll()` - Patch 11 (new): Pulled out from patch 13 (prev. 11) - Patch 12 (new): Suggested by Eric - Patch 13 (prev. 11): - Drop false polling handlers - Use qemu_fcntl_addfl() instead of fcntl(F_SETFL) to keep pre-existing FD flags - Add a note that the buffer returned by read needs to be freed via qemu_vfree() - Pulled out a variable rename into the new patch 11 - Patch 15 (prev. 13): - Simplified the co_read_from_fuse_fd() interface thanks to no longer needing to support poll handlers - Increment in-flight counter before entering the coroutine to make it more obvious how tihs ensures that the export pointer remains valid throughout - Patch 16 (new): Add a common multi-threading interface for exports instead of a specific one just for FUSE - Patch 17 (new): Test cases for this new interface - Patch 18 (prev. 14): - Drop the contrasting with virtio-blk from the commit message; explaining the interface is no longer necessary now that it’s introduced separately in patch 16. - Generally, the interface definition is removed in favor of the new one in patch 16. - Some rebase conflicts (due to other changes earlier in this series). - Patch 19 (new): Stefan suggested adding an explicit note for users on how multi-threading behaves with FUSE, not least because in the future this behavior may depend on the specific implementation features chosen (io-uring or not). Because the actual multi-thread interface is now on the common export options, it is no longer obvious where to put this implementation note; I decided to put it into the general description of the FUSE export options, inside of this dedicated patch. - Patch 20 (new): Simple sanity test for FUSE multi-threading (just test that e.g. nothing crashes when running qemu-img bench on top) - Patch 21 (prev. 15): Rebase conflict due to the changes in patch 15; kept Stefan’s R-b anyway Review notes/suggestions I deliberately did not follow in v2: - Stefan suggested to make patch 1 simpler and more robust by allocating a new buffer for each request. This is indeed a simple change (for patch 1) that I wouldn’t mind, and that I also started to implement. However, eventually I decided against it: The problem doesn’t disappear with the rest of the series, it basically stays the exact same; though instead of an implicit aio_poll() leading to nested polling, it turns into an implicit coroutine yield doing pretty much the same. For performance, it is better not to allocate a new buffer for each request; we only really need a bounce buffer for writes, as there is no other case where we’d continue to read the request buffer after yielding. Therefore, the final state I would like to have after this series is to use a common request buffer for all requests on a single queue, only using a bounce buffer for writes. With that, I think it’s better to implement exactly that right from the start, rather than introducing a new intermediate state. git-backport-diff from v1: Key: [----] : patches are identical [####] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively 001/21:[0012] [FC] 'fuse: Copy write buffer content before polling' 002/21:[----] [--] 'fuse: Ensure init clean-up even with error_fatal' 003/21:[----] [--] 'fuse: Remove superfluous empty line' 004/21:[----] [--] 'fuse: Explicitly set inode ID to 1' 005/21:[----] [--] 'fuse: Change setup_... to mount_fuse_export()' 006/21:[----] [--] 'fuse: Fix mount options' 007/21:[----] [--] 'fuse: Set direct_io and parallel_direct_writes' 008/21:[----] [--] 'fuse: Introduce fuse_{at,de}tach_handlers()' 009/21:[----] [--] 'fuse: Introduce fuse_{inc,dec}_in_flight()' 010/21:[----] [--] 'fuse: Add halted flag' 011/21:[down] 'fuse: Rename length to blk_len in fuse_write()' 012/21:[down] 'block: Move qemu_fcntl_addfl() into osdep.c' 013/21:[0077] [FC] 'fuse: Manually process requests (without libfuse)' 014/21:[----] [--] 'fuse: Reduce max read size' 015/21:[0061] [FC] 'fuse: Process requests in coroutines' 016/21:[down] 'block/export: Add multi-threading interface' 017/21:[down] 'iotests/307: Test multi-thread export interface' 018/21:[0077] [FC] 'fuse: Implement multi-threading' 019/21:[down] 'qapi/block-export: Document FUSE's multi-threading' 020/21:[down] 'iotests/308: Add multi-threading sanity test' 021/21:[0002] [FC] 'fuse: Increase MAX_WRITE_SIZE with a second buffer' Hanna Czenczek (21): fuse: Copy write buffer content before polling fuse: Ensure init clean-up even with error_fatal fuse: Remove superfluous empty line fuse: Explicitly set inode ID to 1 fuse: Change setup_... to mount_fuse_export() fuse: Fix mount options fuse: Set direct_io and parallel_direct_writes fuse: Introduce fuse_{at,de}tach_handlers() fuse: Introduce fuse_{inc,dec}_in_flight() fuse: Add halted flag fuse: Rename length to blk_len in fuse_write() block: Move qemu_fcntl_addfl() into osdep.c fuse: Manually process requests (without libfuse) fuse: Reduce max read size fuse: Process requests in coroutines block/export: Add multi-threading interface iotests/307: Test multi-thread export interface fuse: Implement multi-threading qapi/block-export: Document FUSE's multi-threading iotests/308: Add multi-threading sanity test fuse: Increase MAX_WRITE_SIZE with a second buffer qapi/block-export.json | 39 +- include/block/export.h | 12 +- include/qemu/osdep.h | 1 + block/export/export.c | 48 +- block/export/fuse.c | 1181 ++++++++++++++++++++------ block/export/vduse-blk.c | 7 + block/export/vhost-user-blk-server.c | 8 + block/file-posix.c | 17 +- nbd/server.c | 6 + util/osdep.c | 18 + tests/qemu-iotests/307 | 47 + tests/qemu-iotests/307.out | 18 + tests/qemu-iotests/308 | 55 +- tests/qemu-iotests/308.out | 61 +- 14 files changed, 1213 insertions(+), 305 deletions(-) -- 2.49.0
