On Thu, 29 Oct 2020 04:53:09 -0700, Arnd Bergmann wrote: > > On Thu, Oct 29, 2020 at 11:07 AM Vincent Whitchurch > <vincent.whitchu...@axis.com> wrote: > > > > On Wed, Oct 28, 2020 at 04:50:36PM +0100, Arnd Bergmann wrote: > > > I think we should try to do something on top of the PCIe endpoint > > > subsystem > > > to make it work across arbitrary combinations of host and device > > > implementations, > > > and provide a superset of what the MIC driver, (out-of-tree) Bluefield > > > endpoint > > > driver, and the NTB subsystem as well as a couple of others used to do, > > > each of them tunneling block/network/serial/... over a PCIe link of some > > > sort, usually with virtio. > > > > VOP is not PCIe-specific (as demonstrated by the vop-loopback patches I > > posted a while ago [1]), and it would be a shame for a replacement to be > > tied to the PCIe endpoint subsystem. There are many SOCs out there > > which have multiple Linux-capable processors without cache-coherency > > between them. VOP is (or should I say was since I guess it's being > > deleted) the closest we have in mainline to easily get generic virtio > > (and not just rpmsg) running between these kind of Linux instances. If > > a new replacement framework were to be PCIe-exclusive then we'd have to > > invent one more framework for non-PCIe links to do pretty much the same > > thing. > > > > [1] > > https://lore.kernel.org/lkml/20190403104746.16063-1-vincent.whitchu...@axis.com/ > > Right, sorry I forgot about that. I think this means we should keep having > an abstraction between VOP (under whichever name) and the lower levels, > and be aware that it might run on any number of these: > > - PCIe endpoint, with the endpoint controlling the virtio configuration > - PCIe endpoint, with the host (the side that has the pci_driver) controlling > the virtio configuration > - NTB connections > - your loopback mode > - Virtio tunnels between VM guests (see https://www.linaro.org/projects/#STR) > - Intel MIC (to be removed, but it would be wrong to make assumptions that > cannot be made on that type of hardware)
A virtio interface being one between host and guest is inherently asymmetric. The whole innovation of the VOP design was to treat Linux on a PCIe device as a guest, there was even talk at some point of the "guest" being managed via libvirt. So here host and guest retain their specific role/personality. The host "inserts" devices which appear in the guest e.g. So I am not sure how this asymmetry plays in the scenarios mentioned above.
