On Fri, Jun 19, 2015 at 11:24 AM, Daniele Di Proietto <diproiet...@vmware.com> wrote: > > > On 18/06/2015 23:57, "Traynor, Kevin" <kevin.tray...@intel.com> wrote: > >> >> >>> -----Original Message----- >> >>> From: dev [mailto:dev-boun...@openvswitch.org] On Behalf Of Daniele Di >> >>> Proietto >> >>> Sent: Tuesday, June 16, 2015 7:39 PM >> >>> To: dev@openvswitch.org >> >>> Subject: [ovs-dev] [PATCH] dpif-netdev: Check for PKT_RX_RSS_HASH flag. >> >>> >> >>> DPDK mbufs contain a valid RSS hash only if PKT_RX_RSS_HASH is >> >>> set in 'ol_flags'. Otherwise the hash is garbage and doesn't >> >>> relate to the packet. >> >>> >> >>> This fixes an issue with vhost, which, being a virtual NIC, doesn't >> >>> compute the hash. >> >>> >> >>> Unfortunately the ixgbe vPMD doesn't set the PKT_RX_RSS_HASH, forcing >> >>> OVS to compute an hash is software. This has a significant impact on >> >>> performance (-30% throughput in a single flow setup) which can be >> >>> mitigated in the CPU supports crc32c instructions. >> >> >> >>As per the other thread on this I'm a bit concerned about the performance >> >>drop from this patch, so I did some testing of this and alternative/ >> >>complimentary solutions. >> >> >> >>Here's the options I looked at and some comments: >> >>1. This patch in isolation: vhost drops about ~15% vhost-vhost and >> >>phy-vhost-phy (because of sw hash) but also there is drops of ~25% for >> >>phy-phy and ~15% drop for phy-ivshmem-phy. >> >> >> >>2. Leave the code as is and let EMC misses happen for vhost rx pkts: >> >>I measure this at ~35% drop if missed *everytime* for vhost-vhost. We >> >>see in testing that it can also never happen, but this is not realistic. >> >>There should be no impact to other DPDK interfaces. >> >> >> >>3. Add hash reset for packets from vhost: This is another way of forcing >> >>the software hash for vhost rx and it is roughly equivalent in performance >> >>to 1. for vhost-vhost (~15% drop). While there is a no significant drop >> >>for phy-vhost-phy. There should be no impact to other DPDK interfaces. >> >> >> >>4. Apply this patch and turn off Rx Vectorisation. vhost-vhost will drop >> >>~15% as per 1. and there should be nothing significant for phy-vhost-phy. >> >>We would lose the 10% gain that rx vectorisation gave us for phy-phy. >> >>There should be no impact for dpdkr ports. >> >> >> >> >> >>In terms of not knowing whether the hw hash is valid or not if the flag is >> >>not checked, I would have expected the pmd to return an error on config if >> >>the hash wasn't supported, but I'm not sure that it does. >> >>In the worst case where there was an incorrect hash, it would miss the EMC >> >>which is about a 45% drop for phy-phy. I would think it's pretty safe that >> >>if we configure it, the hash will be correct but I guess there is a >> >>possibility it wouldn't be. >> >> >> >>Even if it is possible to get a smaller patch to fix the underlying issue >> >>in DPDK, it would be in DPDK 2.1 at the earliest meaning the performance >> >>would remain low until sometime in August. If it's DPDK 2.2, then it would >> >>be sometime in December. This would mean any performance drops would be >> >>present in OVS 2.4 and possibly OVS 2.5. >> >> >> >>Sorry :( but based on the performance drop with this patch in isolation it >> >>would be a NAK from me. My preference would be 3 which gives best >>performance, >> >>or 4 which is a bit lower for phy-phy but safer. >> >> >> >>Kevin. > > Thanks for all the testing. I guess it might make sense to stretch our > interpretation of the API in this case, because it wouldn't affect > correctness. > > Unless there any other objection I'm fine with the 3rd approach. >
We can use 3rd approach to fix issue on branch 2.4. Then have patch to check the PKT_RX_RSS_HASH flag on master. By the time we release branch 2.5 we will have proper fix in DPDK and performance will bounce back. _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
