I have found out the problem is related to using iommu=pt and a device not setting dma_mask_coherent.
In that case the kernel does not use the identity mapping but a specific one for that device. But a kernel bug avoids the device mapping being removed if iommu=pt whatever the device gets or not the identity mapping. I can see that has been fixed at some point because last kernel has not this problem. On Tue, Apr 21, 2015 at 6:38 PM, Alejandro Lucero < alejandro.luc...@netronome.com> wrote: > Hi, > > I've been trying to understand some DMAR errors we got with our device > which supports SRIOV. We create 8 VFs and we use two of them for > configuring two ethernet interfaces. No VM is involved. The first time > interfaces are upped all works fine. Some time later they are downed and > the VFs removed. If we create the VFs again then we got this when upping > the interfaces: > > [ 611.152374] dmar: DMAR:[DMA Read] Request device [02:08.0] fault addr > fffe8000 > [ 611.152374] DMAR:[fault reason 06] PTE Read access is not set > [ 611.324461] dmar: DMAR:[DMA Read] Request device [02:08.1] fault addr > fffe8000 > [ 611.324461] DMAR:[fault reason 06] PTE Read access is not set > > The address involved is just the first dma address used by the device for > getting RX ring descriptors. I have used iommu kernel traces and also some > specific ones (as I found those events not helpful enough) and I could see > the mappings being done both times the devices are upped and with the right > read/write permissions. > > Then I looked at the work done each time the VFs are created and I found > out the context entry is not created the second time. When VFs are removed > the iommu_bus_notifier gets action for removing device calling > intel_iommu_remove_device. There just call for removing the iommu group is > done so the context entry is used later on. > > I have seen that more work is done when a device is detached from a VM. In > that case code calls to clear_context_entry and also some calls to flushing > iotlb. So I have tried adding the same code when removing the VF and this > solves the issue. > > I do not know if this is the right thing to do. I know attaching and > detaching devices to VMs is likely more usual and then that functionality > is well tested. By other hand, creating and destroying VFs could be > unlikely or even considered unsense and then this has never been tested. > > I'm working with a 3.13.0-49 Ubuntu kernel. Last stable kernel from Linus > Torvalds is not doing anything different related to the remove device > action. > > The patch applied to intel-iommu.c is attached. > > >
_______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
