Hi, > -----Original Message----- > From: Jesse Gross [mailto:je...@kernel.org] > Sent: Friday, March 25, 2016 7:57 PM > To: Wojciechowicz, RobertX <robertx.wojciechow...@intel.com> > Cc: ovs dev <dev@openvswitch.org> > Subject: Re: [ovs-dev] [PATCH RFC] netdev-dpdk: Expand RSS hash > calculation by MAC addresses > > On Fri, Mar 25, 2016 at 7:45 AM, Wojciechowicz, RobertX > <robertx.wojciechow...@intel.com> wrote: > > Hi Jesse, > > > > please find my answers inline. > > > >> -----Original Message----- > >> From: Jesse Gross [mailto:je...@kernel.org] > >> Sent: Thursday, March 24, 2016 11:20 PM > >> To: Wojciechowicz, RobertX <robertx.wojciechow...@intel.com> > >> Cc: ovs dev <dev@openvswitch.org> > >> Subject: Re: [ovs-dev] [PATCH RFC] netdev-dpdk: Expand RSS hash > >> calculation by MAC addresses > >> > >> On Thu, Mar 24, 2016 at 7:30 AM, Robert Wojciechowicz > >> <robertx.wojciechow...@intel.com> wrote: > >> > This patch configures RSS hash filter in DPDK in order to include > >> > source and destination MAC addresses into RSS hash calculation. > >> > This configuration considerably improves performance for flows > >> > with overlapping IP address spaces and non-overlapping Ethernet > >> > address spaces. > >> > Here is the example test result (throughput in frames per second) > >> > I have done for 16 flows (fixed IP addresses and changing source > >> > MAC addresses) on X710 for 10GbE SFP+: > >> > > >> > 1. original hash calculation: 4870667 fps > >> > 2. expanded hash calculation (with this patch): 12740258 fps > >> > > >> > Signed-off-by: Robert Wojciechowicz > <robertx.wojciechow...@intel.com> > >> > >> This is somewhat of an unusual configuration - I guess as evidenced by > >> the fact that it is not supported by ixgbe. > >> > >> How much of a difference does this make in non-synthetic benchmarks? I > >> would expect in most cases that the use of L4 ports would provide > >> sufficient entropy even in cases where there are overlapping IP > >> addresses. > >> > > > > In most cases you're probably right, but there are use cases where > > this overlapping exists. For example in openstack configurations > > with many tenants. > > The idea of this patch is to make this configuration possible > > for people who really need it. > > I understand that that there can be configurations where there are > overlapping IP addresses, however, I'm not sure that really means that > the L4 ports aren't able to still spread the load. Have you seen real > world situations where this patch makes a difference in practice, as > opposed to just in theory? >
We contacted our customer regarding this real world test, but unfortunately we're still waiting for response. However synthetic benchmarks could be useful also, e.g. for comparison with other vswitches. I discussed this issue also with openstack team in more detail. It seems that overlapping ip and mac address spaces can happen for vlan or other tunnels they use a lot in openstack setups. So there is suggestion to expand this hash by tunnel id instead of mac addresses. I will try to test this scenario also. > The reason why some of the RSS modes you are enabling here aren't on > by default is usually because they can introduce reordering. So I'm > not really eager to go down this path unless you can show that there > are specific reasons why the assumptions behind the default > configuration don't apply here. > To be honest I'm not sure how expanding rss hash may introduce reordering. My understanding is that hash filter is a mechanism to statistically distribute received packets into several receive queues. By default there is used 5-tuple (src_addr, dst_addr, protocol, src_port, dst_port). This patch tries to expand this tuple by mac or tunnel_id in order to reduce collisions for some traffic flows and improve packets distribution into queues. Could you please explain why it may introduce reordering? > >> Two other concerns: > >> * This appears to enable hashing for a wide range of packets, > >> including IP fragments, which would typically allow the possibility of > >> reordering. I think we probably want to be a little more careful in > >> considering what to include. > >> * I think we need to more gracefully handle NICs where this > >> configuration is not supported. I don't like that we should what looks > >> like an error message in normal conditions. > > > > Unfortunately DPDK returns EINVAL error also for unsupported operations, > > so I couldn't handle it in more reasonable way. I wanted to make it > > minimally invasive for other hardware than XL710, so I report all errors > > as info messages. > > Basically I'm not sure if there is some recommended approach > > for providing functionality to OVS, which is hardware specific? > > If there is any I could of course reimplement this patch appropriately. > > I don't think that we have any hardware specific configuration in OVS. > The best way to handle this is to fix DPDK or expose an API that > abstracts out the hardware specific pieces. Ok, so DPDK patch could be the next step in this work. Br, Robert _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
