On Tue, Mar 03, 2026 at 12:56:46PM -0500, Demi Marie Obenour wrote: > Spectrum (https://spectrum-os.org) is going to be implementing > virtio devices outside of the host. One proposed method of doing > this is virtio-vhost-user, which is a virtio device that allows a > VM to expose a vhost-user device to another VM. For instance, one > could assign a NIC to one VM and have it provide a vhost-user-net > device for use by a different VM. > > I brought this up on the KVM/QEMU community call today. Alex Bennée > recommended using virtio-msg instead. However, I have a few concerns > with this: > > 1. Virtio-msg buses are specific to a given hypervisor or (in the > case of FF-A) to a given CPU architecture. None of the current > buses support KVM on platforms other than Arm64. Therefore, > a brand-new bus would be needed.
Hi Demi, The virtio-msg AMP PCI bus works on KVM both for ARM and x86 and any other arch, it's generic. These are the patches we posted to qemu-devel: https://lore.kernel.org/qemu-devel/[email protected]/ > > 2. Virtio-msg requires not-yet-upstream drivers in both the frontend > (the VM using the device) and the backend (the VM providing the > device). Vhost-user uses any of the existing transports, such as > PCI or MMIO. This means that upstream drivers can be used in the > frontend, and also enables supports for Windows and other guests > that lack support for virtio-msg. Fair point, a bit of chicken and egg... > > 3. Vhost-user is already widely deployed, so frontend implementations > are quite well tested. A KVM-specific virtio-msg transport would > serve only one purpose: driver VMs (with assigned devices) on > non-Arm64 platforms. This is a quite niche use-case. Therefore, > I'm concerned that the needed frontend code will be poorly tested > and bitrot. > > Manos Pitsidianakis stated that vhost-user does not make sense in > this case. Why is that? Would it make sense to use virtio-msg > between VMM and its VM, and expose a vhost-user device to the > outside world? What about having the virtio-vhost-user guest driver > emulate a virtio-msg transport, so that it can be used with any device > implementation supporting virtio-msg? > > I would greatly appreciate any and all suggestions here. This is a > serious project that is going to be used in production, but I want > to ensure that the design is the best possible. I've not looked in detail at virtio-vhost-user but it seems to be a bit similar to virtio-msg over PCI AMP with some differences. My take is: vhost-user is an interface designed for splitting virtio transport from virtio device backend in user-space on the same OS instance. The virtio-vhost-user device, enables the use of vhost-user across VM's streching vhost-user's intended use. This is a great step but Since vhost-user is not a virtio transport, It is not end-to-end in the sense that you need QEMU to translate virtio-pci to vhost-user and tunnel it over the virtio-vhost-user device to the backend. Virtio-msg is from the start a transport meant to work between heterogenous systems, different OS, OS instances even different SoCs. It's end-to-end in the sense that if you have a front-end driver in the front-end VM with an appropriate virtio-msg bus you can talk directly with the backend without intermediate proxies, potentially without VM exits. Cheers, Edgar
