With current reset scheme, DMA capable devices are reset before the vIOMMU which translate them. This holds for the different IOMMUs and various DMA capable devices such as virtio devices and VFIO ones. With virtio devices spurious traces can be observed at qemu level such as "virtio: zero sized buffers are not allowed" while for VFIO devices, translation faults can be observed at host level. Virtio devices use 3 phase reset and virtio-pci devices are reset in the 'hold' phase. VFIO device reset are registered using qemu_register_reset() and as a consequence they are also reset on 'hold' phase.
Note that the tree of QOM devices resets depth-first but it does so while enforcing the 3 phases. First the tree is traversed doing the 'enter' phase, then the 'hold' phase and eventually the 'exit' phase. However the QOM hierarchy is not built so that vIOMMUs get reset before the DMA capable devices (IOMMUs are using either legacy reset scheme or hold phase). Changing the QOM hierarchy does not sound trivial while forcing the vIOMMUs to be reset on 'exit' phase sounds reasonable and much simpler. Obviously this relies on the assumption that all DMA capable devices quiesce their DMA before (ie. during 'enter' or hold' phase). This was tested with qmp system_reset and virsh reset. Best Regards Eric This series can be found at: https://github.com/eauger/qemu/tree/viommu-3phase-reset-v2 History: v1 -> v2: - Removed hw/i386/intel_iommu: Tear down address spaces before IOMMU reset - Also move SMMU base class reset to exit reset. This was an oversight from v1 - Add last patch documenting expectations in terms of DMA reset - Improved commit messages and cover letter - dared to keep Michael's A-b for the patches whose code was not altered References: [1] [PATCH 0/4] intel_iommu: Reset vIOMMU after all the rest of devices https://lore.kernel.org/all/20240117091559.144730-1-pet...@redhat.com/ Eric Auger (5): hw/virtio/virtio-iommu: Migrate to 3-phase reset hw/i386/intel-iommu: Migrate to 3-phase reset hw/arm/smmuv3: Move reset to exit phase hw/vfio/common: Add a trace point in vfio_reset_handler docs/devel/reset: Document reset expectations for DMA and IOMMU docs/devel/reset.rst | 5 +++++ hw/arm/smmu-common.c | 9 +++++++-- hw/arm/smmuv3.c | 14 ++++++++++---- hw/i386/intel_iommu.c | 12 +++++++++--- hw/vfio/common.c | 1 + hw/virtio/virtio-iommu.c | 14 ++++++++++---- hw/arm/trace-events | 1 + hw/i386/trace-events | 1 + hw/vfio/trace-events | 1 + hw/virtio/trace-events | 2 +- 10 files changed, 46 insertions(+), 14 deletions(-) -- 2.47.1
