> -----Original Message----- > From: Jean-Philippe Brucker [mailto:jean-philippe.bruc...@arm.com] > Sent: Thursday, July 06, 2017 3:33 PM > To: Bharat Bhushan <bharat.bhus...@nxp.com>; Auger Eric > <eric.au...@redhat.com>; eric.auger....@gmail.com; > peter.mayd...@linaro.org; alex.william...@redhat.com; m...@redhat.com; > qemu-...@nongnu.org; qemu-devel@nongnu.org > Cc: w...@redhat.com; kevin.t...@intel.com; marc.zyng...@arm.com; > t...@semihalf.com; will.dea...@arm.com; drjo...@redhat.com; > robin.mur...@arm.com; christoffer.d...@linaro.org > Subject: Re: [Qemu-devel] [RFC v2 0/8] VIRTIO-IOMMU device > > On 05/07/17 09:49, Bharat Bhushan wrote:>>> Also when setup msi-route > kvm_irqchip_add_msi_route() we needed to > >> provide the translated address. > >>> According to my understanding this is required because kernel does > >>> no go > >> through viommu translation when generating interrupt, no? > >> > >> yes this is needed when KVM MSI routes are set up, ie. along with GICV3 > ITS. > >> With GICv2M, qemu direct gsi mapping is used and this is not needed. > >> > >> So I do not understand your previous sentence saying "MSI interrupts > >> works without any change". > > > > I have almost completed vfio integration with virtio-iommu and now > > testing the changes by assigning e1000 device to VM. For this I have > > changed virtio-iommu driver to use IOMMU_RESV_MSI rather than sw-msi > > and this does not need changed in vfio_get_addr() and > > kvm_irqchip_add_msi_route() > > I understand you're reserving region 0x08000000-0x08100000 as > IOMMU_RESV_MSI instead of IOMMU_RESV_SW_MSI? I think this only > works because Qemu places the vgic in that area as well (in hw/arm/virt.c). > It's not a coincidence if the addresses are the same, because Eric chose them > for the Linux SMMU drivers and I copied them. > > We can't rely on that behavior, though, it will break MSIs in emulated > devices. And if Qemu happens to move the MSI doorbell in future machine > revisions, then it would also break VFIO.
Yes, make sense to me > > Just for my own understanding -- what happens, I think, is that in Linux > iova_reserve_iommu_regions initially reserves the guest-physical doorbell > 0x08000000-0x08100000. Then much later, when the device driver requests > an MSI, the irqchip driver calls iommu_dma_map_msi_msg with the guest- > physical gicv2m address 0x08020000. The function finds the right page in > msi_page_list, which was added by cookie_init_hw_msi_region, therefore > bypassing the viommu and the GPA gets written in the MSI-X table. This means in case tomorrow when qemu changes virt machine address map and vgic-its (its-translator register address) address range does not fall in the msi_page_list then it will allocate a new iova, create mapping in iommu. So this will no longer be identity mapped and fail to work with new qemu? > > If an emulated device such as virtio-net-pci were to generate an MSI, then > Qemu would attempt to access the doorbell written by Linux into the MSI-X > table, 0x08020000, and fault because that address wasn't mapped in the > viommu. > > So for VFIO, you either need to translate the MSI-X entry using the viommu, > or just assume that the vaddr corresponds to the only MSI doorbell > accessible by this device (because how can we be certain that the guest > already mapped the doorbell before writing the entry?) > > For ARM machines it's probably best to stick with IOMMU_RESV_SW_MSI. > However, a nice way to use IOMMU_RESV_MSI would be for the virtio- > iommu device to advertise identity-mapped/reserved regions, and bypass > translation on these regions. Then the driver could reserve those with > IOMMU_RESV_MSI. Correct me if I did not understood you correctly, today iommu-driver decides msi-reserved region, what if we change this and virtio-iommu device will provide the reserved msi region as per the emulated machine (virt/intel). So virtio-iommu driver will use the address advertised by virtio-iommu device as IOMMU_RESV_MSI. In this case msi-page-list will always have the reserved region for MSI. On qemu side, for emulated devices we will let virtio-iommu return same address as translated address as it falls in MSI-reserved page already known to it. > For x86 we will need such a system, with an added IRQ > remapping feature. I do not understand x86 MSI interrupt generation, but If above understand is correct, then why we need IRQ remapping for x86? Will the x86 machine emulated in QEMU provides a big address range for MSIs and when actually generating MSI it needed some extra processing (IRQ-remapping processing) before actually generating write transaction for MSI interrupt ? Thanks -Bharat > > Thanks, > Jean
