One more point, if you're not doing this already: Allocate 2^N-1 mbufs, not 2^N. According to the code and comments: "The optimum size (in terms of memory usage) for a mempool is when n is a power of two minus one: n = (2^q - 1)."
The reason: rte_mempool_create(... n ...) invokes rte_ring_create(... rte_align32power(n+1) ...), i.e., it wants to allocate n+1 slots in the ring, but it has to round that up to the next power of 2. So, when you allocate 2^N bufs, its ring comes with twice as many slots (2^(N+1)). -- Robert On Fri, Sep 20, 2013 at 2:48 AM, Dmitry Vyal <dmitryvyal at gmail.com> wrote: > On 09/19/2013 11:39 PM, Robert Sanford wrote: > >> Hi Dmitry, >> >> The biggest drop-off seems to be from size 128K to 256K. Are you using >> 1GB huge pages already (rather than 2MB)? >> >> I would think that it would not use over 1GB until you ask for 512K mbufs >> or more. >> >> > Hi Robert, > > Yes, I've been using 1GB pages for a while. My L3 cache is 20MB and mbufs > are 2240 bytes of size. So something strange indeed happens then we move > from ~200MB to ~400MB. Any ideas? > > Regards, > Dmitry > >
