> -----Original Message-----
> From: Christoph Hellwig <h...@infradead.org>
> Sent: Monday, June 9, 2025 12:35 PM
> To: Christian König <christian.koe...@amd.com>
> Cc: wangtao <tao.wang...@honor.com>; Christoph Hellwig
> <h...@infradead.org>; sumit.sem...@linaro.org; kra...@redhat.com;
> vivek.kasire...@intel.com; v...@zeniv.linux.org.uk; brau...@kernel.org;
> hu...@google.com; a...@linux-foundation.org; amir7...@gmail.com;
> benjamin.gaign...@collabora.com; brian.star...@arm.com;
> jstu...@google.com; tjmerc...@google.com; j...@suse.cz;
> baolin.w...@linux.alibaba.com; linux-me...@vger.kernel.org; dri-
> de...@lists.freedesktop.org; linaro-mm-...@lists.linaro.org; linux-
> ker...@vger.kernel.org; linux-fsde...@vger.kernel.org; linux-
> m...@kvack.org; wangbintian(BintianWang) <bintian.w...@honor.com>;
> yipengxiang <yipengxi...@honor.com>; liulu 00013167
> <liulu....@honor.com>; hanfeng 00012985 <feng....@honor.com>
> Subject: Re: [PATCH v4 0/4] Implement dmabuf direct I/O via
> copy_file_range
>
> On Fri, Jun 06, 2025 at 01:20:48PM +0200, Christian König wrote:
> > > dmabuf acts as a driver and shouldn't be handled by VFS, so I made
> > > dmabuf implement copy_file_range callbacks to support direct I/O
> > > zero-copy. I'm open to both approaches. What's the preference of VFS
> > > experts?
> >
> > That would probably be illegal. Using the sg_table in the DMA-buf
> > implementation turned out to be a mistake.
>
> Two thing here that should not be directly conflated. Using the sg_table was
> a huge mistake, and we should try to move dmabuf to switch that to a pure
I'm a bit confused: don't dmabuf importers need to traverse sg_table to
access folios or dma_addr/len? Do you mean restricting sg_table access
(e.g., only via iov_iter) or proposing alternative approaches?
> dma_addr_t/len array now that the new DMA API supporting that has been
> merged. Is there any chance the dma-buf maintainers could start to kick this
> off? I'm of course happy to assist.
>
> But that notwithstanding, dma-buf is THE buffer sharing mechanism in the
> kernel, and we should promote it instead of reinventing it badly.
> And there is a use case for having a fully DMA mapped buffer in the block
> layer and I/O path, especially on systems with an IOMMU.
> So having an iov_iter backed by a dma-buf would be extremely helpful.
> That's mostly lib/iov_iter.c code, not VFS, though.
Are you suggesting adding an ITER_DMABUF type to iov_iter, or
implementing dmabuf-to-iov_bvec conversion within iov_iter?
>
> > The question Christoph raised was rather why is your CPU so slow that
> > walking the page tables has a significant overhead compared to the
> > actual I/O?
>
> Yes, that's really puzzling and should be addressed first.
With high CPU performance (e.g., 3GHz), GUP (get_user_pages) overhead
is relatively low (observed in 3GHz tests).
| 32x32MB Read 1024MB |Creat-ms|Close-ms| I/O-ms|I/O-MB/s| I/O%
|---------------------------|--------|--------|--------|--------|-----
| 1) memfd direct R/W| 1 | 118 | 312 | 3448 | 100%
| 2) u+memfd direct R/W| 196 | 123 | 295 | 3651 | 105%
| 3) u+memfd direct sendfile| 175 | 102 | 976 | 1100 | 31%
| 4) u+memfd direct splice| 173 | 103 | 443 | 2428 | 70%
| 5) udmabuf buffer R/W| 183 | 100 | 453 | 2375 | 68%
| 6) dmabuf buffer R/W| 34 | 4 | 427 | 2519 | 73%
| 7) udmabuf direct c_f_r| 200 | 102 | 278 | 3874 | 112%
| 8) dmabuf direct c_f_r| 36 | 5 | 269 | 4002 | 116%
With lower CPU performance (e.g., 1GHz), GUP overhead becomes more
significant (as seen in 1GHz tests).
| 32x32MB Read 1024MB |Creat-ms|Close-ms| I/O-ms|I/O-MB/s| I/O%
|---------------------------|--------|--------|--------|--------|-----
| 1) memfd direct R/W| 2 | 393 | 969 | 1109 | 100%
| 2) u+memfd direct R/W| 592 | 424 | 570 | 1884 | 169%
| 3) u+memfd direct sendfile| 587 | 356 | 2229 | 481 | 43%
| 4) u+memfd direct splice| 568 | 352 | 795 | 1350 | 121%
| 5) udmabuf buffer R/W| 597 | 343 | 1238 | 867 | 78%
| 6) dmabuf buffer R/W| 69 | 13 | 1128 | 952 | 85%
| 7) udmabuf direct c_f_r| 595 | 345 | 372 | 2889 | 260%
| 8) dmabuf direct c_f_r| 80 | 13 | 274 | 3929 | 354%
Regards,
Wangtao.
