On 02/24/2014 10:51 AM, Prarit Bhargava wrote: > The ixgbe driver makes some assumptions about the layout of cpus in the > system which are not always correct given a particular system layout. The > ixgbe driver allocates one MSI/cpu for queue usage but the code does not take > into account that devices are located on NUMA nodes and that the cpus in a > node > are not contiguous. > > These issues were found while doing cpu hotplug testing, however, both of > these > issues can lead to obvious system performance issues as they defeat the > purpose of having one MSI processing a queue per cpu. > > Cc: Jeff Kirsher <[email protected]> > Cc: Jesse Brandeburg <[email protected]> > Cc: Bruce Allan <[email protected]> > Cc: Carolyn Wyborny <[email protected]> > Cc: Don Skidmore <[email protected]> > Cc: Greg Rose <[email protected]> > Cc: Alex Duyck <[email protected]> > Cc: John Ronciak <[email protected]> > Cc: Mitch Williams <[email protected]> > Cc: "David S. Miller" <[email protected]> > Cc: [email protected] > Cc: [email protected] > Cc: [email protected] > > Prarit Bhargava (2): > ixgbe, make interrupt allocations NUMA aware > ixgbe, don't assume mapping of numa node cpus > > drivers/net/ethernet/intel/ixgbe/ixgbe.h | 2 ++ > drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c | 44 > ++++++++++++++++++------ > drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 6 ++-- > drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c | 5 +-- > 4 files changed, 42 insertions(+), 15 deletions(-) >
This is a step in the right direction but totally defeats the purpose of ATR. With this change we might as well defeature ATR all together since things are now back to RSS w/ NUMA specific allocations which is what we had a couple of years ago. The code as it is written now would be a better for for igb which doesn't have ATR than ixgbe. ATR is supposed to map 1:1 queues to CPUs. The problem is RSS is also a factor and not especially smart or NUMA aware. The ideal solution would be to allocate the first N CPUs, where N is the number in the local node for ATR/RSS. Then map all other queues as ATR with a 1:1 mapping to CPUs. Thanks, Alex ------------------------------------------------------------------------------ Flow-based real-time traffic analytics software. Cisco certified tool. Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer Customize your own dashboards, set traffic alerts and generate reports. Network behavioral analysis & security monitoring. All-in-one tool. http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk _______________________________________________ E1000-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
