On Fri, 18 Oct 2019 06:08:31 +0000 "Boeuf, Sebastien" <sebastien.bo...@intel.com> wrote:
> Hi folks, > > We have been recently implementing a nested VFIO solution for our Cloud > Hypervisor VMM. Thanks to virtio-iommu, we can now pass a device > through nested virtualization. > > After some performances testing, we realized the device in L2 was > slightly less performant than the same device in L1. The DMAR has > nothing to do with it since it is programmed only at boot time with the > entire guest RAM mapping for each device, but the problem comes from > the interrupts. When using polling instead of interrupts, we can > clearly see the device in L2 behaving very closely to what is expected > in L1. > > So basically, because the interrupt has to bounce through each layer of > virtualization, the more layers the less performance we will get out of > this device, right? Yes > We were wondering if there was any on going work to allow posted > interrupts to be delivered directly to the last layer of virtualization > where the device is actually used. I might be missing something but in > theory, it seems feasible, the problem seems to be that VFIO does not > update the IRT of the IOMMU, so I don't know how we could manually > update it, the same way we can update the DMAR with VFIO_MAP_DMA ioctl > for instance. > And the second part is about knowing which interrupt needs to be > updated in the IRT, which means the virtio-iommu device should be able > to provide those kind of information. > > It seems like there are some possibilities but I clearly don't have the > whole picture in mind, and I'm interested in learning more about any > existing work on this topic. I agree with your assessment, it seems like it should be possible, but I don't know of any work to support it. Note that vfio doesn't directly manipulate posted interrupts, it's done via the irqbypass producer/consumer mechanism, so perhaps this largely needs to be done in KVM. I can't say I've spent many cycles thinking about it either though. Thanks, Alex _______________________________________________ vfio-users mailing list vfio-users@redhat.com https://www.redhat.com/mailman/listinfo/vfio-users
