On Sat, 21 Jan 2017 22:31:26 +0200 Saeed Mahameed <sae...@dev.mellanox.co.il> wrote:
> > My previous measurements show approx 20℅ speedup on a UDP test with delivery > > to remote CPU. > > > > Removing the cache would of cause be a good usecase for speeding up the page > > allocator (PCP). Which Mel Gorman and me are working on. AFAIK current page > > order0 cost 240 cycles. Mel have reduced til to 180, and without NUMA 150 > > cycles. And with bulking this can be amortized to 80 cycles. > > > > Are you trying to say that we won't need the cache if you manage to > deliver those optimizations ? These page alloc optimization are good, but not fast-enough to replace your cache. Maybe I can improve it further, but it will be very hard to compete with a recycle cache (it can skip many checks the page alloc need, as it knows the page state). Said in another way, the gap will be significantly smaller, and you will not see a big boost from using this cache. BUT there are other advantages of using a guaranteed recycle pool facility (like the page_pool). Namely, (1) DMA-overhead: keeping page DMA mapped to counter DMA+IOMMU overhead, (2) RX-zero-copy: opens up for a software memory model solution for pre-VMA-mapping pages to userspace (See: [1] for argument how this avoids leaking kernel mem, but only expose/leak packet-data mem) > can you compare those optimizations with the page_frag_cache from > dev_alloc_skb ? IHMO the page_frag_cache is indirectly a bulk page allocator facility, which is limiting the number of times the page allocator is called, and thus amortize the cost of talking to the page allocator. Performance wise, it is likely comparable to your page cache, and likely faster than the 80 cycles order0-bulking. BUT we generally worry about using these 32K pages, as it opens a window for attackers pinning down memory. [1] https://prototype-kernel.readthedocs.io/en/latest/vm/page_pool/design/memory_model_nic.html -- Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer
