Hi Nathan,
On 11/22/24 7:53 PM, Nathan Chen wrote: > > >> Also as a heads up, I've added support for auto-inserting PCIe > switch > > >> between the PXB and GPUs in libvirt to attach multiple devices to a > > SMMU > > >> node per libvirt's documentation - "If you intend to plug multiple > > >> devices into a pcie-expander-bus, you must connect a > > >> pcie-switch-upstream-port to the pcie-root-port that is plugged > into the > > >> pcie-expander-bus, and multiple pcie-switch-downstream-ports to the > > >> pcie-switch-upstream-port". Future unit-tests should follow this > > >> topology configuration. > > > > > > > Ok. Could you please give me an example Qemu equivalent command > > > option, > > > > if possible, for the above case. I am not that familiar with > libvirt > > > and I would > > > > also like to test the above scenario. > > > > > > You can use "-device x3130-upstream" for the upstream switch port, > and > > > "-device xio3130-downstream" for the downstream port: > > > > > > -device pxb-pcie,bus_nr=250,id=pci.1,bus=pcie.0,addr=0x1 \ > > > -device pcie-root-port,id=pci.2,bus=pci.1,addr=0x0 \ > > > -device x3130-upstream,id=pci.3,bus=pci.2,addr=0x0 \ > > > -device xio3130- > > > downstream,id=pci.4,bus=pci.3,addr=0x0,chassis=17,port=1 \ > > > -device vfio-pci,host=0009:01:00.0,id=hostdev0,bus=pci.4,addr=0x0 \ > > > -device arm-smmuv3-nested,pci-bus=pci.1 > > > > Thanks. Just wondering why libvirt mandates usage of pcie-switch for > multiple > > device plugging rather than just using pcie-root-ports? > > > > Please let me if there is any advantage in doing so that you are > aware > of. > > Actually it seems like that documentation I quoted is out of date. > That section of the documentation for pcie-expander-bus was written > before a patch that revised libvirt's pxb to have 32 slots instead of > just 1 slot, and it wasn't updated afterwards. you mean read QEMU documentation in qemu/docs/pcie.txt (esp PCI Express only hierarchy) Thanks Eric > > With your branch and my libvirt prototype, I was still able to attach > a passthrough device behind a PCIe switch and see it attached to a > vSMMU in the VM, so I'm not sure if you need to make additional > changes to your solution to support this. But I think we should still > support/test the case where VFIO devices are behind a switch, > otherwise we're placing a limitation on end users who have a use case > for it. > > -Nathan
