On Wed, Feb 05, 2025 at 12:53:42PM -0800, Nathan Chen wrote: > > > On 1/31/2025 8:08 AM, Eric Auger wrote: > > > > > > > And Qemu does some checking to make sure that the device is indeed > > > > > > associated > > > > > > > with the specified phys-smmuv3. This can be done going through > > > > > > > the > > > > > > sysfs path checking > > > > > > > which is what I guess libvirt is currently doing to populate the > > > > topology. > > > > > > So basically > > > > > > > Qemu is just replicating that to validate again. > > > > > > I would prefer that iommufd users not have to go out to sysfs.. > > > > > > > > > > > > > Or another option is extending the IOMMU_GET_HW_INFO IOCTL to > > > > > > return the phys > > > > > > > smmuv3 base address which can avoid going through the sysfs. > > > > > > It also doesn't seem great to expose a physical address. But we > > > > > > could > > > > > > have an 'iommu instance id' that was a unique small integer? > > > > > Ok. But how the user space can map that to the device? > > > > Why does it need to? > > > > > > > > libvirt picks some label for the vsmmu instance, it doesn't matter > > > > what the string is. > > > > > > > > qemu validates that all of the vsmmu instances are only linked to PCI > > > > device that have the same iommu ID. This is already happening in the > > > > kernel, it will fail attaches to mismatched instances. > > > > > > > > Nothing further is needed? > > > -device pxb-pcie,id=pcie.1,bus_nr=8,bus=pcie.0 \ > > > -device pcie-root-port,id=pcie.port1,bus=pcie.1,chassis=1 \ > > > -device arm-smmuv3-accel,bus=pcie.1,id=smmuv1 \ > > I don't get what is the point of adding such an id if it is not > > referenced anywhere? > > > > Eric > > Daniel mentions that the host-to-guest SMMU pairing must be chosen such that > it makes conceptual sense w.r.t. the guest NUMA to host NUMA pairing [0]. > The current implementation allows for incorrect host to guest numa node > pairings, e.g. pSMMU has affinity to host numa node 0, but it’s paired with > a vSMMU paired with a guest numa node pinned to host numa node 1. > > By specifying the host SMMU id, we can explicitly pair a host SMMU with a > guest SMMU associated with the correct PXB NUMA node, vs. implying the > host-to-guest SMMU pairing based on what devices are attached to the PXB. > While it would not completely prevent the incorrect pSMMU/vSMMU pairing > w.r.t. host to guest numa node pairings, specifying the pSMMU id would make > the implications of host to guest numa node pairings more clear when > specifying a vSMMU instance.
You've not specified any host SMMU id in the above CLI args though, only the PXB association. It needs something like -device arm-smmuv3-accel,bus=pcie.1,id=smmuv1,host-smmu=XXXXX where 'XXXX' is some value to identify the host SMMU With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
