Hi Stefano,
Stefano Stabellini <sstabell...@kernel.org> writes: > + Stewart, Vikram > > On Wed, 15 Nov 2023, Oleksandr Tyshchenko wrote: >> On 15.11.23 14:33, Julien Grall wrote: >> > Thanks for adding support for virtio-pci in Xen. I have some questions. >> > >> > On 15/11/2023 11:26, Sergiy Kibrik wrote: >> >> From: Oleksandr Tyshchenko <oleksandr_tyshche...@epam.com> >> >> >> >> In order to enable more use-cases such as having multiple >> >> device-models (Qemu) running in different backend domains which provide >> >> virtio-pci devices for the same guest, we allocate and expose one >> >> PCI host bridge for every virtio backend domain for that guest. >> > >> > OOI, why do you need to expose one PCI host bridge for every stubdomain? >> > >> > In fact looking at the next patch, it seems you are handling some of the >> > hostbridge request in Xen. This is adds a bit more confusion. >> > >> > I was expecting the virtual PCI device would be in the vPCI and each >> > Device emulator would advertise which BDF they are covering. >> >> >> This patch series only covers use-cases where the device emulator >> handles the *entire* PCI Host bridge and PCI (virtio-pci) devices behind >> it (i.e. Qemu). Also this patch series doesn't touch vPCI/PCI >> pass-through resources, handling, accounting, nothing. From the >> hypervisor we only need a help to intercept the config space accesses >> happen in a range [GUEST_VIRTIO_PCI_ECAM_BASE ... >> GUEST_VIRTIO_PCI_ECAM_BASE + GUEST_VIRTIO_PCI_TOTAL_ECAM_SIZE] and >> forward them to the linked device emulator (if any), that's all. >> >> It is not possible (with current series) to run device emulators what >> emulate only separate PCI (virtio-pci) devices. For it to be possible, I >> think, much more changes are required than current patch series does. >> There at least should be special PCI Host bridge emulation in Xen (or >> reuse vPCI) for the integration. Also Xen should be in charge of forming >> resulting PCI interrupt based on each PCI device level signaling (if we >> use legacy interrupts), some kind of x86's XEN_DMOP_set_pci_intx_level, >> etc. Please note, I am not saying this is not possible in general, >> likely it is possible, but initial patch series doesn't cover these >> use-cases) >> >> We expose one PCI host bridge per virtio backend domain. This is a >> separate PCI host bridge to combine all virtio-pci devices running in >> the same backend domain (in the same device emulator currently). >> The examples: >> - if only one domain runs Qemu which servers virtio-blk, virtio-net, >> virtio-console devices for DomU - only single PCI Host bridge will be >> exposed for DomU >> - if we add another domain to run Qemu to serve additionally virtio-gpu, >> virtio-input and virtio-snd for the *same* DomU - we expose second PCI >> Host bridge for DomU >> >> I am afraid, we cannot end up exposing only single PCI Host bridge with >> current model (if we use device emulators running in different domains >> that handles the *entire* PCI Host bridges), this won't work. > > > We were discussing the topic of vPCI and Virtio PCI just this morning > with Stewart and Vikram. We also intend to make them work well together > in the next couple of months (great timing!!) > > However, our thinking is to go with the other approach Julien > suggested: a single PCI Root Complex emulated in Xen by vPCI. QEMU would > register individual PCI devices against it. > > Vikram, Stewart, please comment. Our understanding is that it should be > possible to make QEMU virtio-pci work with vPCI with relatively minor > efforts and AMD volunteers to do the work in the next couple of months > on the vPCI side. > > > Although it should be possible to make both approaches work at the same > time, given that it would seem that EPAM and AMD have very similar > requirements, I suggest we work together and collaborate on a single > approach going forward that works best for everyone. > > > Let me start by saying that if we can get away with it, I think that a > single PCI Root Complex in Xen would be best because it requires less > complexity. Why emulate 2/3 PCI Root Complexes if we can emulate only > one? Well, in fact we tried similar setup, this was in the first version of virtio-pci support. But we had a couple of issues with this. For instance, this might conflict with PCI passthrough devices, with virtio devices that have back-ends in different domains, etc. I am no saying that this is impossible, but this just involves more moving parts. With my vPCI patch series in place, hypervisor itself assigns BDFs for passed-through devices. Toolstack needs to get this information to know which BDFs are free and can be used by virtio-pci. -- WBR, Volodymyr
