Hi all: As the userspace vitio driver became popular, more and more request were received for secure DMA environemt (DMAR). So this series tries to make DMAR works for virtio/vhost. The idea is let virtio/vhost co-work with userspace iommu implememtation. This is done through:
- for virtio, when platform supports IOMMU (VIRTIO_F_IOMMU_PLATFORM), virtio will not assume address_space_memory, instead a transport specific method were introduced for querying the dma address space and dma helpers were used in device emulation codes. - for vhost, implement a device IOTLB by using device IOTLB API supported by recent kernel. With this API, vhost kernel can query IOTLB entry for a specified iova from qemu, qemu can invalidate an arbitrary range of iova in vhost kernel. The device IOTLB API is totaly architecture independent, an example implementation was done with intel iommu by: - implement basic ATS (Address Translation Service) for virtio-pci, this will make device IOTLB visible for iommu driver in guest. - implement device IOTLB descriptor processing in intel iommu (enabled through device-iotlb=on), and trigger the device IOTLB invalidation in vhost through iommu notifier. It could be easily ported to other IOMMU or architecture even if it doesn't support device IOTLB. (e.g just invalidate the vhost IOTLB during IOMMU IOTLB invalidation). But this will be slow since several devics were contending userspace IOTLB entries. AMD IOMMU suppot for device IOTLB is ready, but it depends on other fixes to work correctly. Test was done by: - intel_iommu=on/strict in guest. - vfio l2fwd in guest. This main use case is the programs that use fixed mapping in guest (e.g dpdk). If 1G hugepage were used in guest, thanks to the SLLPS support, we can get 100% TLB hit rate for l2fwd in guest. For the normal kernel driver which uses lots of dynamic mapping and unmapping, we may see performance penalty, this could be optimized in the future. TODO: - more platforms and IOMMU support (done but block by other bugs of AMD IOMMU) - performance optimizations (e.g merging adjacent mappings) - non ATS support (userspace IOTLB snooping) - using ATSCtl to disable Device IOTLB Changes from V3: - rebase to HEAD - fill all entry of IOMMUTLBEntry before notify Changes from V2: - rebase to HEAD - avoid querying dma_as each time by using vdev->dma_as directly - fix and improve address_space_get_iotlb_entry() - drop patch 1 (which has been posted as an independent fix) - fix ECAP when device-iotlb=off - fix centos6 build Changes from V1: - rebase to HEAD - avoid calling transport specific dma as fetching method each time by caching it in vdev - convert to use new IOMMU notifier API - silent checkpatch warnings and fix 32bit build - use "device-iotlb" instead of "device_iotlb" - rename virtio_memory_map() to vhost_memory_map() and move it to vhost.c - use memory_region_is_iommu() instead of inventing new one Changes from RFC: - rebase to HEAD - switch to use new vhost device IOTLB API - use the new feature bit VIRITO_F_IOMMU_PLATFORM - finalize basic ATS implementation - add ATSR for Root port ATS transaction - fix the iommu notifier handling during unregistering - use snprintf() in patch 3 - correc the loop in address_space_get_iotlb_entry() - small tweak on the address calculation during device iotlb descriptor processing. Jason Wang (10): virtio: convert to use DMA api intel_iommu: name vtd address space with devfn intel_iommu: allocate new key when creating new address space exec: introduce address_space_get_iotlb_entry() intel_iommu: support device iotlb descriptor virtio-pci: address space translation service (ATS) support acpi: add ATSR for q35 memory: handle alias for iommu notifier memory: handle alias in memory_region_is_iommu() vhost_net: device IOTLB support exec.c | 33 ++++++ hw/block/virtio-blk.c | 2 +- hw/char/virtio-serial-bus.c | 3 +- hw/i386/acpi-build.c | 9 ++ hw/i386/intel_iommu.c | 92 +++++++++++++-- hw/i386/intel_iommu_internal.h | 13 ++- hw/i386/x86-iommu.c | 17 +++ hw/pci/pcie.c | 16 +++ hw/scsi/virtio-scsi.c | 4 +- hw/virtio/vhost-backend.c | 99 +++++++++++++++++ hw/virtio/vhost.c | 178 ++++++++++++++++++++++++++---- hw/virtio/virtio-bus.c | 8 ++ hw/virtio/virtio-pci.c | 21 ++++ hw/virtio/virtio-pci.h | 4 + hw/virtio/virtio.c | 57 ++++++---- include/exec/memory.h | 8 ++ include/hw/acpi/acpi-defs.h | 12 ++ include/hw/i386/x86-iommu.h | 1 + include/hw/pci/pcie.h | 4 + include/hw/virtio/vhost-backend.h | 13 +++ include/hw/virtio/vhost.h | 4 + include/hw/virtio/virtio-access.h | 31 ++++-- include/hw/virtio/virtio-bus.h | 1 + include/hw/virtio/virtio.h | 9 +- include/standard-headers/linux/pci_regs.h | 1 + memory.c | 9 ++ net/tap.c | 1 + 27 files changed, 582 insertions(+), 68 deletions(-) -- 2.7.4
