On Tue, Mar 23, 2021 at 04:55:31PM +0000, Alexander Lobakin wrote: > > > > > >
[...] > > > > > > > > > > Thanks for the testing! > > > > > Any chance you can get a perf measurement on this? > > > > > > > > I guess you mean perf-report (--stdio) output, right? > > > > > > > > > > Yea, > > > As hinted below, I am just trying to figure out if on Alexander's > > > platform the > > > cost of syncing, is bigger that free-allocate. I remember one armv7 were > > > that > > > was the case. > > > > > > > > Is DMA syncing taking a substantial amount of your cpu usage? > > > > > > > > (+1 this is an important question) > > Sure, I'll drop perf tools to my test env and share the results, > maybe tomorrow or in a few days. > From what I know for sure about MIPS and my platform, > post-Rx synching (dma_sync_single_for_cpu()) is a no-op, and > pre-Rx (dma_sync_single_for_device() etc.) is a bit expensive. > I always have sane page_pool->pp.max_len value (smth about 1668 > for MTU of 1500) to minimize the overhead. > > By the word, IIRC, all machines shipped with mvpp2 have hardware > cache coherency units and don't suffer from sync routines at all. > That may be the reason why mvpp2 wins the most from this series. Yep exactly. It's also the reason why you explicitly have to opt-in using the recycling (by marking the skb for it), instead of hiding the feature in the page pool internals Cheers /Ilias > > > > > > > > > > > > > [0] > > > > > > https://lore.kernel.org/netdev/20210323153550.130385-1-aloba...@pm.me > > > > > > > > > > > > > > That would be the same as for mvneta: > > > > Overhead Shared Object Symbol > > 24.10% [kernel] [k] __pi___inval_dcache_area > > 23.02% [mvneta] [k] mvneta_rx_swbm > > 7.19% [kernel] [k] kmem_cache_alloc > > > > Anyway, I tried to use the recycling *and* napi_build_skb on mvpp2, > > and I get lower packet rate than recycling alone. > > I don't know why, we should investigate it. > > mvpp2 driver doesn't use napi_consume_skb() on its Tx completion path. > As a result, NAPI percpu caches get refilled only through > kmem_cache_alloc_bulk(), and most of skbuff_head recycling > doesn't work. > > > Regards, > > -- > > per aspera ad upstream > > Oh, I love that one! > > Al >
