On Fri, Jan 13, 2023 at 10:57:00AM -0700, Alex Williamson wrote: > On Fri, 13 Jan 2023 12:39:18 +0000 > Jean-Philippe Brucker <jean-phili...@linaro.org> wrote: > > > Hi, > > > > On Mon, Jan 09, 2023 at 10:11:19PM +0100, Eric Auger wrote: > > > > Jean, do you have any idea about how to fix that? Do you think we have a > > > > trouble in the acpi/viot setup or virtio-iommu probe sequence. It looks > > > > like virtio probe and attach commands are called too early, before the > > > > bus is actually correctly numbered. > > > > > > So after further investigations looks this is not a problem of bus > > > number, which is good at the time of the virtio cmd calls but rather a > > > problem related to the devfn (0 was used when creating the IOMMU MR) > > > whereas the virtio-iommu cmds looks for the non aliased devfn. With that > > > fixed, the probe and attach at least succeeds. The device still does not > > > work for me but I will continue my investigations and send a tentative > > > fix. > > > > If I remember correctly VIOT can deal with bus numbers because bridges are > > assigned a range by QEMU, but I haven't tested that in detail, and I don't > > know how it holds with conventional PCI bridges. > > In my reading of the virtio-iommu spec,
Hm, is that the virtio-iommu spec or ACPI VIOT/device tree spec? The virtio-iommu spec shouldn't refer to PCI buses at the moment. The intent is that for PCI, the "endpoint ID" passed in an ATTACH request corresponds to PCI segment and RID of PCI devices at the time of the request (so after the OS renumbered the buses). If you found something in the spec that contradicts this, it should be fixed. Note that "endpoint" is a misnomer, it can refer to PCI bridges as well, anything that can issue DMA transactions. > I noted that it specifies the > bus numbers *at the time of OS handoff*, so it essentially washes its > hands of the OS renumbering buses while leaving subtle dependencies on > initial numbering in the guest and QEMU implementations. Yes we needed to describe in the firmware tables (device-tree and ACPI VIOT) which devices the IOMMU manages. And at the time we generate the tables, if we want to refer to PCI devices behind bridges, we can either use catch-all ranges for any possible bus numbers they will get, or initialize bus numbers in bridges and pass those to the OS. But that's only to communicate the IOMMU topology to the OS, because we couldn't come up with anything better. After it sets up PCI the OS should be able to use its own configuration of the PCI topology in virtio-iommu requests. > On bare metal, a conventional bridge aliases the devices downstream of > it. We reflect that in QEMU by aliasing those devices to the > AddressSpace of the bridge. IIRC, Linux guests will use a > for-each-dma-alias function when programming IOMMU translation tables > to populate the bridge alias, where a physical IOMMU would essentially > only see that bridge RID. Thanks, Yes there might be something missing in the Linux driver, I'll have a look Thanks, Jean
