On Fri, 30 Nov 2018 23:56:34 +0000 Nikos Dragazis <ndraga...@outlook.com.gr> wrote:
> Hi all, > > I am trying to understand how vfio enables DMA from user space memory. I > would appreciate any help on this. > > Suppose we have a system with x86 architecture and an IOMMU. When a user > space driver in that system wants to ask from a PCI device to perform > DMA from user space memory, it has to register that memory with > VFIO_IOMMU_MAP_DMA ioctl. What are the restrictions that this user space > memory has to meet? How are the DMA memory limitations met? Is this > totally arranged by vfio? Or is it a responsibility of the user space > process (eg. use hugepages)? > > I had a conversation on the SPDK mailing list about this. SPDK is a > storage framework running entirely in user space and uses vfio to > control the PCI storage devices. This is the relevant email: > https://lists.01.org/pipermail/spdk/2018-November/002733.html > > I am a little bit confused about the purpose of hugepages in SPDK and > whether they are related to vfio. > > To summarize, I am trying to find the answer to the following questions: > > Is there any constraint about the DMA memory buffers passed to vfio? They need to match the minimum page size and alignment requirements of the IOMMU, ex. 4K on x86. The user specified IOVA needs to fit within the physical limits of the IOMMU, typically 39 or 48 bits on Intel systems. > Is it possible to pass a buffer allocated with malloc? Yep. > Or is it necessary to use hugepages which by design serve all the DMA > memory limitations (pinned, non-swappable, physically contiguous)? Not required, vfio will pin pages itself, physically discontiguous is handled transparently in the mapping ioctl. Using hugepages will see benefits that we'll forward those contiguous mappings to the IOMMU, which can then use superpages in the IOMMU page tables. Regarding your "DMA memory limitations" question above, I'm not sure if this is referring to limited DMA address space of the device, but the user specifies the IOVA, so it's up to them to specify an address reachable by the device if such a limit exists. Thanks, Alex _______________________________________________ vfio-users mailing list vfio-users@redhat.com https://www.redhat.com/mailman/listinfo/vfio-users
