On Tue, Oct 21, 2014 at 09:07:40AM -0600, Alex Williamson wrote: > On Tue, 2014-10-21 at 13:15 +0200, Alexander Graf wrote: > > > > On 21.10.14 11:35, Knut Omang wrote: > > > On Tue, 2014-10-21 at 11:07 +0200, Alexander Graf wrote: > > >> > > >> > > >>> Am 21.10.2014 um 07:26 schrieb Knut Omang <knut.om...@oracle.com>: > > >>> > > >>>> On Tue, 2014-10-21 at 01:29 +0200, Alexander Graf wrote: > > >>>> > > >>>> > > >>>>> Am 21.10.2014 um 00:34 schrieb Knut Omang <knut.om...@oracle.com>: > > >>>>> > > >>>>> This patch set changes the data structure used to handle address > > >>>>> spaces within > > >>>>> the emulated Intel iommu to support traversal also if bus numbers are > > >>>>> dynamically > > >>>>> allocated, as is the case for devices that sit behind root ports or > > >>>>> downstream switches. > > >>>>> This means that we cannot use bus number as index, instead a QLIST is > > >>>>> used. > > >>>>> > > >>>>> This requires a change in the API for setup of IOMMUs which is taken > > >>>>> care of by > > >>>>> the first patch. The second patch implements the fix. > > >>>> > > >>>> Are you sure that this works on real hardware? How does that one > > >>>> communicate sub-bridge liodns to the iommu? How do they get indexed > > >>>> from software? > > >>> > > >>> I do not claim to fully understand the details of how this is > > >>> implemented in hardware, but I believe the implementation I propose here > > >>> should be functionally equivalent to what the Intel IOMMU offers, and > > >>> similar to the original implementation here, except that the data > > >>> structure is valid also before enumeration when behind buses. > > >> > > >> Can you please give me a pointer to the vt-d spec's section that > > >> explains iommu behavior behind bridges? > > >> > > >> I've also added Alex W who has played with PCI bridges behind iommus > > >> quite a bit recently. > > >> > > >>> > > >>> After enumeration, the only difference would be that during > > >>> invalidation, there is a list search for the right bus rather than an > > >>> index lookup as before, slightly less efficient but at the benefit of > > >>> being independent of bus numbering during setup. > > >> > > >> I don't think the implementation is bad, I'm just not sure that it > > >> follows the spec, > > >> so I want to confirm :). > > > > > > http://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/vt-directed-io-spec.pdf > > > > So if I understand that document correctly, a PCIe / PCI-X bridge can > > swizzle the requester id depending on a device behind itself. PCI > > bridges can not - there everything behind the bridge will appear as if > > the DMA originated from the bridge device. > > > > So conceptually, PCIe / PCI-X bridges should probably be the ones > > converting requester IDs. > > PCIe-to-PCI/X bridges alias requester IDs to the subordinate bus, so all > requests appear to come from subordinate-bus-number:00.0. PCI-X does > support a requester ID, but there are numerous rules where the bridge > can take ownership of the transaction that require the IOMMU to handle > bridges as an alias of the device. PCI bridges alias all downstream > devices as the bridge itself, but if you look at > quirk_use_pcie_bridge_dma_alias() in the kernel, there are numerous > cases where bridges behave like a PCIe-to-PCI bridge, but fail to > include a PCIe capability. Thanks, > > Alex
In fact, apprently exactly 4 such bridges exist. We don't emulate them nor do we emulate pci-x so it's pretty simple I think. -- MST
