On Thu, Mar 27, 2025 at 11:55:57AM -0400, Stefan Hajnoczi wrote:
> On Tue, Mar 25, 2025 at 05:06:54PM +0100, Hanna Czenczek wrote:
> > FUSE allows creating multiple request queues by "cloning" /dev/fuse FDs
> > (via open("/dev/fuse") + ioctl(FUSE_DEV_IOC_CLONE)).
> >
> > We can use this to implement multi-threading.
> >
> > Note that the interface presented here differs from the multi-queue
> > interface of virtio-blk: The latter maps virtqueues to iothreads, which
> > allows processing multiple virtqueues in a single iothread. The
> > equivalent (processing multiple FDs in a single iothread) would not make
> > sense for FUSE because those FDs are used in a round-robin fashion by
> > the FUSE kernel driver. Putting two of them into a single iothread will
> > just create a bottleneck.
>
> This text might be outdated. virtio-blk's new iothread-vq-mapping
> parameter provides the "array of iothreads" mentioned below and a way to
> assign virtqueues to those IOThreads.
>
> > +++ b/qapi/block-export.json
> > @@ -179,12 +179,18 @@
> > # mount the export with allow_other, and if that fails, try again
> > # without. (since 6.1; default: auto)
> > #
> > +# @iothreads: Enables multi-threading: Handle requests in each of the
> > +# given iothreads (instead of the block device's iothread, or the
> > +# export's "main" iothread). For this, the FUSE FD is duplicated so
> > +# there is one FD per iothread. (since 10.1)
>
> This option isn't FUSE-specific but FUSE is the first export type to
> support it. Please add it to BlockExportOptions instead and refuse
> export creation when the export type only supports 1 IOThread.
>
> Eric: Are you interested in implementing support for multiple IOThreads
> in the NBD export? I remember some time ago we talked about NBD
> multi-conn support, although maybe that was for the client rather than
> the server.
The NBD server already supports clients that make requests through
multiple TCP sockets, but right now, that server is not taking
advantage of iothreads to spread the TCP load.
And yes, I am in the middle of working on adding client NBD multi-conn
support (reviving Rich Jones' preliminary patches on what it would
take to have qemu open parallel TCP sockets to a supporting NBD
server), which also will use a round-robin approach (but here, the
round-robin is something we would code up in qemu, rather than the
behavior handed to us by the FUSE kernel layer). Pinning specific
iothreads to a specific TCP socket may or may not make sense, but I
definitely want to have support for handing a pool of iothreads to an
NBD client that will be using multi-conn.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc.
Virtualization: qemu.org | libguestfs.org
