On Fri, Feb 2, 2024, at 9:05 PM, Rick Macklem wrote: > > But the page size is only 4K on most platforms. So while an M_EXTPGS mbuf > > can hold 5 pages (..from memory, too lazy to do the math right now) and > > reduces socket buffer mbuf chain lengths by a factor of 10 or so (2k vs 20k > > per mbuf), the S/G list that a NIC will need to consume would likely > > decrease only by a factor of 2. And even then only if the busdma code to > > map mbufs for DMA is not coalescing adjacent mbufs. I know busdma does > > some coalescing, but I can't recall if it coalesces physcally adjacent > > mbufs. > > I'm guessing the factor of 2 comes from the fact that each page is a > contiguous segment?
Actually, no, I'm being dumb. I was thinking that pages would be split up, but that's wrong. Without M_EXTPGS, each mbuf generated by sendfile (or nfs) would be an M_EXT with a wrapper around a single 4K page. So the scatter/gather list would be exactly the same. The win would be if the pages themselves were contiguous (which they often are), and if the bus_dma mbuf mapping code coalesced those segments, and if the device could handle DMA across a 4K boundary. That's what would get you shorter s/g lists. I think tcp_m_copy() can handle this now, as if_hw_tsomaxsegsize is set by the driver to express how long the max contiguous segment they can handle is. BTW, I really hate the mixing of bus dma restrictions with the hw_tsomax stuff. It always makes my head explode.. Drew
