> -----Original Message----- > From: Daniele Di Proietto [mailto:diproiet...@vmware.com] > Sent: Friday, April 10, 2015 2:01 PM > To: Traynor, Kevin > Cc: Pravin Shelar; dev@openvswitch.org > Subject: Re: [ovs-dev] [PATCH] netdev-dpdk: Change eth rx burst size. > > > > On 10 Apr 2015, at 00:22, Traynor, Kevin <kevin.tray...@intel.com> wrote: > > > > > >> -----Original Message----- > >> From: Pravin Shelar [mailto:pshe...@nicira.com] > >> Sent: Thursday, April 9, 2015 10:50 PM > >> To: Traynor, Kevin > >> Cc: Daniele Di Proietto; dev@openvswitch.org > >> Subject: Re: [ovs-dev] [PATCH] netdev-dpdk: Change eth rx burst size. > >> > >> On Thu, Apr 9, 2015 at 2:36 PM, Traynor, Kevin <kevin.tray...@intel.com> > >> wrote: > >>> > >>>> -----Original Message----- > >>>> From: Pravin Shelar [mailto:pshe...@nicira.com] > >>>> Sent: Thursday, April 9, 2015 9:10 PM > >>>> To: Traynor, Kevin; Daniele Di Proietto > >>>> Cc: dev@openvswitch.org > >>>> Subject: Re: [ovs-dev] [PATCH] netdev-dpdk: Change eth rx burst size. > >>>> > >>>> On Wed, Apr 8, 2015 at 8:43 AM, Kevin Traynor <kevin.tray...@intel.com> > >>>> wrote: > >>>>> Change eth rx burst size from 192 to 32. This significantly > >>>>> improves performance for packets that will be forwarded > >>>>> through dpdkvhost ports, as the max dpdkvhost tx burst > >>>>> size (32) will not be exceeded. There are negligible > >>>>> effects in other scenarios. > >>>>> > >>>>> Signed-off-by: Kevin Traynor <kevin.tray...@intel.com> > >>>> > >>>> Daniele, > >>>> You mentioned that you are going to fix the issue by handling > >>>> different burst size of vhost devices. Do you have the fix? > >>> > >>> It wasn't clear in the commit message but just to clarify, the > >>> dpdkvhost max burst size is set in the DPDK vhost library and if > >>> we try and enqueue >32 packets only 32 will be enqueued. We can > >>> add a back-off/retry but it will give better performance to > >>> ensure that the vhost burst size is not exceeded. > >>> > >> > >> How about adding for loop in vhost_send() to handle burst of packets > >> larger than 32? > > > > Yeah, the DPDK sample app uses a backoff(small sleep)/retry loop but > > you could put it in a busy loop too. I was planning to add some retry > > code anyway as it's possible that you could get bursts of >32 from a > > different interface, so it's good to handle this. > > > > The virtqueues are of very limited size (128 packets I think), so they > > are not ideally suited to bursty traffic which I guess is why DPDK have > > set the max burst size to 32. Time the PMD spends waiting/retrying is > > time it could be doing something else, so I think it is better to reduce > > the burst size where possible. I haven't seen any advantage to the 192 eth > > rx burst size in my testing as changing to 32 I get negligible difference > > on phy2phy tests (~1%) and I get a 2x-3x performance boost for vhost > > loopback tests, but perhaps there are some use cases where it is > > advantageous? > > I agree with you guys that we should use a loop here, > like in dpdk_queue_flush__(). How much slower would that be? > >
I've tested a simple retry loop with the 192 burst size and the performance is very poor. With the retry loop and a burst size of 32, I see good performance. I think this is because with the bigger burst sizes we are hitting limits/contention in the vhost interface and we have to loop until they are resolved. I think we need a retry loop, but it's not a performant replacement for optimizing the burst size where we can at the moment. I can share the code if you want to test it out on your systems. _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
