It might be interesting to see that start/end address of the 256K-item (256 * 1024 * 2240 = 560 MB) mbuf memory pool. Maybe it's the first size that straddles two 1GB pages.
Perhaps you should try a tool that reports cache misses, TLB misses, and related statistics. I don't know much about this area, yet, but this looks like a good starting place: http://software.intel.com/en-us/articles/intel-performance-counter-monitor-a-better-way-to-measure-cpu-utilization -- Regards, Robert On Fri, Sep 20, 2013 at 2:48 AM, Dmitry Vyal <dmitryvyal at gmail.com> wrote: > On 09/19/2013 11:39 PM, Robert Sanford wrote: > >> Hi Dmitry, >> >> The biggest drop-off seems to be from size 128K to 256K. Are you using >> 1GB huge pages already (rather than 2MB)? >> >> I would think that it would not use over 1GB until you ask for 512K mbufs >> or more. >> >> > Hi Robert, > > Yes, I've been using 1GB pages for a while. My L3 cache is 20MB and mbufs > are 2240 bytes of size. So something strange indeed happens then we move > from ~200MB to ~400MB. Any ideas? > > Regards, > Dmitry > >
