Jason Gunthorpe <[email protected]> writes: > On Mon, Jul 13, 2026 at 08:08:14PM +0100, Fuad Tabba wrote: >> On Sun, 12 Jul 2026 at 02:02, Ackerley Tng <[email protected]> wrote: >> > >> > Jason Gunthorpe <[email protected]> writes: >> > >> > > On Thu, May 29, 2025 at 01:34:53PM +0800, Xu Yilun wrote: >> > >> Export vfio dma-buf specific info by attaching vfio_dma_buf_data in >> > >> struct dma_buf::priv. Provide a helper vfio_dma_buf_get_data() for >> > >> importers to fetch these data. Exporters identify VFIO dma-buf by >> > >> successfully getting these data. >> > >> >> > >> VFIO dma-buf supports disabling host access to these exported MMIO >> > >> regions when the device is converted to private. Exporters like KVM >> > >> need to identify this type of dma-buf to decide if it is good to use. >> > >> KVM only allows host unaccessible MMIO regions been mapped in private >> > >> roots. >> > >> >> > >> Export struct kvm * handler attached to the vfio device. This >> > >> allows KVM to do another sanity check. MMIO should only be assigned to >> > >> a CoCo VM if its owner device is already assigned to the same VM. >> > > >> > > This doesn't seem right, it should be encapsulated into the standard >> > > DMABUF API in some way. >> > > >> > >> > I'd like to propose an alternative. I've been working on guest_memfd and >> > new to the world of IO, please help me along! :) >> > >> > It seems like using dmabufs are used a little awkwardly here. IIUC >> > dmabufs were originally meant to expose memory of one device to another >> > device, mostly meant to share memory. Dmabufs do expose MMIO too, for >> > device to device communications. Without virtualization, userspace MMIO >> > would be done by mmap()-ing a VFIO fd and having the userspace program >> > write to the userspace addresses. >> > >> > Before CoCo, device passthrough (MMIO) is mostly handled by mmap()-ing a >> > VFIO fd and setting up the userspace address in a KVM memslot for the >> > guest. >> > >> > With CoCo, is the problem we're solving that we want KVM to know what >> > pfns to set up in stage 2 page tables, but not via userspace addresses? >> > >> > guest_memfd already does that for regular host memory, tracks the >> > private/shared-ness of the memory, tracks which struct kvm the memory >> > belongs to. >> > >> > guest_memfd functions as KVM's bridge to host memory. KVM already can >> > ask guest_memfd for the pfn to map into stage 2 page tables, and already >> > asks guest_memfd for the shared/private state of the memory. guest_memfd >> > already also blocks the host from faulting guest private memory >> > (mmap()-ing is always allowed). >> > >> > >> > Instead of using dmabuf as the intermediary between the MMIO PFNs and >> > KVM, why not use guest_memfd? >> > >> > What if we make guest_memfd accept a VFIO fd, or a dmabuf fd? >> >> This is interesting for pKVM too, provided it covers more than MMIO. >> >> We need guest_memfd to be backable by a dmabuf for ordinary guest memory, not >> only for device MMIO. There is mobile hardware that doesn't tolerate >> scattered >> private memory (DMA engines that can't gather, IOMMU page-table size >> constraints), and a CMA-backed dmabuf heap is the practical way to get >> contiguous memory at runtime. > > Why can't guestmemfd allocate directly from CMA? Allocating struct > page memory through dmabuf just to put it back in a guestmemfd sounds > very ugly to me. >
I'd like to understand this use case better too, is using CMA through dmabuf basically because CMA doesn't really have a direct userspace interface? As in, for HugeTLB there's fd = memfd_create(HUGETLB) and HugeTLBfs fds, but not CMA? I did hope for this API shape to extend beyong VFIO and HugeTLB to anything that has an fd today, but Frank did bring up the counter point that not all memory has an fd. >> HugeTLB doesn't help, it wants boot-time >> reservation. Those pages are struct-page backed, so it's a different problem >> from the non-struct-page MMIO case, and the shared parts still need to be >> GUP-able. > > Isn't dmabuf pretty allergic to mmaping refcounted struct page backed > memory since that wrecks its lifetime model? > >> More important for the API shape: conversions have to work on subsets of >> such a >> region, at page granularity. A pKVM guest doesn't know what backs its >> memory, so >> it will issue share/unshare hypercalls over arbitrary ranges of whatever it >> was >> given. For my future reference, does this mean for the CMA case, some page splitting on conversion to shared (so each page can have its own refcount to track users) will be necessary? (and merging) >> If a dmabuf-backed guest_memfd can only be converted as a whole, we can't >> use it for memory, and the guest can't be taught to care. > > More reasons not to involve DMABUF since guestmemfd already does all > of this... > > Jason
