Hi Eric, I started added replay in virtio-iommu and came across how MSI interrupts with work with VFIO. I understand that on intel this works differently but vsmmu will have same requirement. kvm-msi-irq-route are added using the msi-address to be translated by viommu and not the final translated address. While currently the irqfd framework does not know about emulated iommus (virtio-iommu, vsmmuv3/vintel-iommu). So in my view we have following options: - Programming with translated address when setting up kvm-msi-irq-route - Route the interrupts via QEMU, which is bad from performance - vhost-virtio-iommu may solve the problem in long term
Is there any other better option I am missing? Thanks -Bharat > -----Original Message----- > From: Auger Eric [mailto:eric.au...@redhat.com] > Sent: Friday, June 09, 2017 5:24 PM > To: Bharat Bhushan <bharat.bhus...@nxp.com>; > eric.auger....@gmail.com; peter.mayd...@linaro.org; > alex.william...@redhat.com; m...@redhat.com; qemu-...@nongnu.org; > qemu-devel@nongnu.org; jean-philippe.bruc...@arm.com > Cc: w...@redhat.com; kevin.t...@intel.com; marc.zyng...@arm.com; > t...@semihalf.com; will.dea...@arm.com; drjo...@redhat.com; > robin.mur...@arm.com; christoffer.d...@linaro.org > Subject: Re: [Qemu-devel] [RFC v2 0/8] VIRTIO-IOMMU device > > Hi Bharat, > > On 09/06/2017 13:30, Bharat Bhushan wrote: > > Hi Eric, > > > >> -----Original Message----- > >> From: Auger Eric [mailto:eric.au...@redhat.com] > >> Sent: Friday, June 09, 2017 12:14 PM > >> To: Bharat Bhushan <bharat.bhus...@nxp.com>; > >> eric.auger....@gmail.com; peter.mayd...@linaro.org; > >> alex.william...@redhat.com; m...@redhat.com; qemu-...@nongnu.org; > >> qemu-devel@nongnu.org; jean-philippe.bruc...@arm.com > >> Cc: will.dea...@arm.com; robin.mur...@arm.com; > kevin.t...@intel.com; > >> marc.zyng...@arm.com; christoffer.d...@linaro.org; > >> drjo...@redhat.com; w...@redhat.com; t...@semihalf.com > >> Subject: Re: [RFC v2 0/8] VIRTIO-IOMMU device > >> > >> Hi Bharat, > >> > >> On 09/06/2017 08:16, Bharat Bhushan wrote: > >>> Hi Eric, > >>> > >>>> -----Original Message----- > >>>> From: Eric Auger [mailto:eric.au...@redhat.com] > >>>> Sent: Wednesday, June 07, 2017 9:31 PM > >>>> To: eric.auger....@gmail.com; eric.au...@redhat.com; > >>>> peter.mayd...@linaro.org; alex.william...@redhat.com; > >> m...@redhat.com; > >>>> qemu-...@nongnu.org; qemu-devel@nongnu.org; jean- > >>>> philippe.bruc...@arm.com > >>>> Cc: will.dea...@arm.com; robin.mur...@arm.com; > >> kevin.t...@intel.com; > >>>> marc.zyng...@arm.com; christoffer.d...@linaro.org; > >>>> drjo...@redhat.com; w...@redhat.com; t...@semihalf.com; Bharat > >> Bhushan > >>>> <bharat.bhus...@nxp.com> > >>>> Subject: [RFC v2 0/8] VIRTIO-IOMMU device > >>>> > >>>> This series implements the virtio-iommu device. This is a proof of > >>>> concept based on the virtio-iommu specification written by > >>>> Jean-Philippe > >> Brucker [1]. > >>>> This was tested with a guest using the virtio-iommu driver [2] and > >>>> exposed with a virtio-net-pci using dma ops. > >>>> > >>>> The device gets instantiated using the "-device virtio-iommu-device" > >>>> option. It currently works with ARM virt machine only as the > >>>> machine must handle the dt binding between the virtio-mmio "iommu" > >>>> node and the PCI host bridge node. ACPI booting is not yet supported. > >>>> > >>>> This should allow to start some benchmarking activities against > >>>> pure emulated IOMMU (especially ARM SMMU). > >>> > >>> I am testing this on ARM64 and see below continuous error prints: > >>> > >>> virtio_iommu_translate sid=8 is not known!! > >>> virtio_iommu_translate sid=8 is not known!! > >>> virtio_iommu_translate sid=8 is not known!! > >>> virtio_iommu_translate sid=8 is not known!! > >>> virtio_iommu_translate sid=8 is not known!! > >>> virtio_iommu_translate sid=8 is not known!! > >>> virtio_iommu_translate sid=8 is not known!! > >>> virtio_iommu_translate sid=8 is not known!! > >>> virtio_iommu_translate sid=8 is not known!! > >>> virtio_iommu_translate sid=8 is not known!! > >>> > >>> > >>> Also in guest I do not see device-tree node with virtio-iommu. > >> do you mean the virtio-mmio with #iommu-cells property? > >> > >> This one is created statically by virt machine. I would be surprised > >> if it were not there. Are you using the virt = virt2.10 machine. > >> Machines before do not support its instantiation. > >> > >> Please can you add a printf in hw/arm/virt.c create_virtio_mmio() at > >> the moment when this node is created. Also you can add a printf in > >> bind_virtio_iommu_device() to make sure the binding with the PCI host > >> bridge is added on machine init done. > >> > >> Also worth to check, CONFIG_VIRTIO_IOMMU=y on guest side. > > > > It works on my side. > Great. > > The driver config was disabled and also I was using guest kernel which was > not have deferred-probing. > Yes I did not mention in my cover letter the guest I have been using is based > on Jean-Philippe's branch, featuring deferred IOMMU probing. I I have not > tried yet with an upstream guest. > Now after fixing it works on my side > > I placed some prints to see dma-map are mapping regions in virtio-iommu, > it uses emulated iommu. > > > > I will continue to add VFIO support now on this and more testing !! > > OK. I will do the VFIO integration first on the vsmmuv3 device as I already > prepared the VFIO replay and hopefully we will sync ;-) > > Thanks > > Eric > > > > Thanks > > -Bharat > > > >> > >> Thanks > >> > >> Eric > >> > >>> I am using qemu-tree you mentioned below and iommu-driver patches > >> published by Jean-P. > >>> Qemu command line have additional ""-device virtio-iommu-device". > >>> What > >> I am missing ? > >> > >> > >>> > >>> Thanks > >>> -Bharat > >>> > >>>> > >>>> Best Regards > >>>> > >>>> Eric > >>>> > >>>> This series can be found at: > >>>> https://github.com/eauger/qemu/tree/virtio-iommu-rfcv2 > >>>> > >>>> References: > >>>> [1] [RFC 0/3] virtio-iommu: a paravirtualized IOMMU, [2] [RFC PATCH > >>>> linux] > >>>> iommu: Add virtio-iommu driver [3] [RFC PATCH kvmtool 00/15] Add > >>>> virtio- iommu > >>>> > >>>> History: > >>>> v1 -> v2: > >>>> - fix redifinition of viommu_as typedef > >>>> > >>>> Eric Auger (8): > >>>> update-linux-headers: import virtio_iommu.h > >>>> linux-headers: Update for virtio-iommu > >>>> virtio_iommu: add skeleton > >>>> virtio-iommu: Decode the command payload > >>>> virtio_iommu: Add the iommu regions > >>>> virtio-iommu: Implement the translation and commands > >>>> hw/arm/virt: Add 2.10 machine type > >>>> hw/arm/virt: Add virtio-iommu the virt board > >>>> > >>>> hw/arm/virt.c | 116 ++++- > >>>> hw/virtio/Makefile.objs | 1 + > >>>> hw/virtio/trace-events | 14 + > >>>> hw/virtio/virtio-iommu.c | 623 > >> ++++++++++++++++++++++++++ > >>>> include/hw/arm/virt.h | 5 + > >>>> include/hw/virtio/virtio-iommu.h | 60 +++ > >>>> include/standard-headers/linux/virtio_ids.h | 1 + > >>>> include/standard-headers/linux/virtio_iommu.h | 142 ++++++ > >>>> linux-headers/linux/virtio_iommu.h | 1 + > >>>> scripts/update-linux-headers.sh | 3 + > >>>> 10 files changed, 957 insertions(+), 9 deletions(-) create mode > >>>> 100644 hw/virtio/virtio-iommu.c create mode 100644 > >>>> include/hw/virtio/virtio- iommu.h create mode 100644 > >>>> include/standard- headers/linux/virtio_iommu.h create mode 100644 > >>>> linux-headers/linux/virtio_iommu.h > >>>> > >>>> -- > >>>> 2.5.5 > >>> > >
