[dpdk-dev] How to fight forwarding performance regression on large mempool sizes.

2013-09-20 Thread Robert Sanford
One more point, if you're not doing this already: Allocate 2^N-1 mbufs, not 2^N. According to the code and comments: "The optimum size (in terms of memory usage) for a mempool is when n is a power of two minus one: n = (2^q - 1)." The reason: rte_mempool_create(... n ...) invokes rte_ring_create(.

[dpdk-dev] How to fight forwarding performance regression on large mempool sizes.

2013-09-20 Thread Dmitry Vyal
On 09/19/2013 11:43 PM, Venkatesan, Venky wrote: > Dmitry, > One other question - what version of DPDK are you doing on? > -Venky > It's DPDK-1.3.1-7 downloaded from intel.com. Should I try upgrading?

[dpdk-dev] How to fight forwarding performance regression on large mempool sizes.

2013-09-20 Thread Dmitry Vyal
On 09/19/2013 11:39 PM, Robert Sanford wrote: > Hi Dmitry, > > The biggest drop-off seems to be from size 128K to 256K. Are you using > 1GB huge pages already (rather than 2MB)? > > I would think that it would not use over 1GB until you ask for 512K > mbufs or more. > Hi Robert, Yes, I've been

[dpdk-dev] How to fight forwarding performance regression on large mempool sizes.

2013-09-20 Thread Robert Sanford
It might be interesting to see that start/end address of the 256K-item (256 * 1024 * 2240 = 560 MB) mbuf memory pool. Maybe it's the first size that straddles two 1GB pages. Perhaps you should try a tool that reports cache misses, TLB misses, and related statistics. I don't know much about this ar