On Tue, Feb 6, 2018 at 7:31 PM, Alex Williamson <alex.william...@redhat.com> wrote: > On Tue, 6 Feb 2018 18:59:43 +0200 > Ran Shalit <ransha...@gmail.com> wrote: > >> On Tue, Feb 6, 2018 at 5:41 PM, Alex Williamson >> <alex.william...@redhat.com> wrote: >> > On Tue, 6 Feb 2018 17:20:08 +0200 >> > Ran Shalit <ransha...@gmail.com> wrote: >> > >> >> On Tue, Feb 6, 2018 at 4:56 PM, Alex Williamson >> >> <alex.william...@redhat.com> wrote: >> >> > On Tue, 6 Feb 2018 16:24:06 +0200 >> >> > Ran Shalit <ransha...@gmail.com> wrote: >> >> > >> >> >> Hello, >> >> >> >> >> >> I need to write vfio userspace driver for pci express. >> >> >> I've been searching for some examples of using vfio, but did not find >> >> >> any, >> >> >> Is there a git/examples of userspace application using vfio ? >> >> > >> >> > git://git.qemu.org/qemu.git >> >> > git://dpdk.org/dpdk >> >> > https://github.com/andre-richter/rVFIO >> >> > https://github.com/MicronSSD/unvme >> >> > https://github.com/awilliam/tests >> >> >> >> Thank you. >> >> I am trying to understand how to use it with dma. >> >> I've been checking now the links, but did not find yet in the above >> >> example how to both map DMA memory and read/write from that memory. >> >> I see that the above test git contains tests which map/unmap DMA, but >> >> not reading/writing from that dma memory afterwards (there is a >> >> seperate test which read/write from memory but without dma). >> >> Is that correct or did I miss something ? Is there any example which >> >> shows how to both map and then read/write ? >> > >> > Initiating DMA is device specific, you need to understand the >> > programming model of the device for that. The vfio DMA ioctls only map >> > the buffers through the IOMMU to allow that DMA, they don't control the >> > device. The unit tests in the last link doesn't have any device >> > specific code, they're only performing the mapping without actually >> > initiating device DMA. The QEMU examples also mostly configure DMA but >> > rely on native drivers within the guest VM to program the device to >> > initiate DMA. The rVFIO project is also just a wrapper to make vfio >> > more accessible to scripting languages, so I don't expect actual device >> > driver code there. DPDK and the NVME driver should have actual driver >> > code for the device though. In fact there's also an NVME project >> > within QEMU that makes use of vfio that might have driver code. The >> > simple logic though is that the DMA map ioctl takes a user provided >> > virtual buffer and maps it through the IOMMU at the user provided IOVA >> > (I/O virtual address). It's up to the user to make the device perform >> > DMA using that IOVA. Note that the IOMMU address width is typically >> > less than the user virtual address space, so identity mapping the IOVA >> > to the virtual address is not a good strategy. >> > >> >> I try to see that I understand the tests code. >> Seems that I understand most of it except the following: >> in vfio-pci-device-open-igd.c >> there is : >> ... >> struct vfio_region_info *region; >> struct vfio_info_cap_header *header; >> buf = region = malloc(region_info.argsz); >> ioctl(device, VFIO_DEVICE_GET_REGION_INFO, region)) >> printf("First cap @%x\n", region->cap_offset); >> header = buf + region->cap_offset; >> printf("header ID %x version %x next %x\n", >> header->id, header->version, header->next); >> .... >> I don't understand some stuff here: >> region and bug point to the same place (vfio_region_info struct) , is >> it that (buf + region->cap_offset) is a pointer to virtual address >> which can be accessed just like that (without any allocation)? > > FWIW, this has nothing to do with DMA, this is just the semantics of > the vfio API and these ioctls are only reading capability information > provided by the kernel interface. The malloc is for a structure which > is filled by the ioctl. Part of the return data in that structure is > the cap_offset, which tells us the offset into that structure for the > head of a chain of capabilities, each of which has a > vfio_info_cap_header. The buf variable is simply a void* pointer to > make indexing that capability chain easier. We're only indexing within > the allocated buffer. > >> in dpdk pci_vfio I see that it is accessed in another way, through >> vfio_dev_fd: >> ret = pread64(fd, ®, sizeof(reg), >> VFIO_GET_REGION_ADDR(VFIO_PCI_CONFIG_REGION_INDEX) + >> cap_offset); > > This is an entirely different operation, also not doing DMA. This is > simply reading from PCI config space of the device via the region > designated for this access. Perhaps you might want to start with an > overview of vfio to help you get situated, hopefully this will help: > > https://www.youtube.com/watch?v=WFkdTFTOTpA >
I understand that these last questions were not related to DMA. I wanted to make sure I understand the access in BARs, as I see in the test code. Sorry for switching between subjects. What seems to confused me is that struct vfio_region_info has no vfio_info_cap_header* field, yet we access vfio_info_cap_header in that struct (struct vfio_region_info) , i.e. header points somewhere inside struct vfio_region_info to vfio_info_cap_header. Best Regards, ran > Thanks, > Alex _______________________________________________ vfio-users mailing list vfio-users@redhat.com https://www.redhat.com/mailman/listinfo/vfio-users
