On Mon, Apr 13, 2015 at 10:09 AM, Traynor, Kevin <kevin.tray...@intel.com> wrote: > >> -----Original Message----- >> From: Daniele Di Proietto [mailto:diproiet...@vmware.com] >> Sent: Friday, April 10, 2015 2:01 PM >> To: Traynor, Kevin >> Cc: Pravin Shelar; dev@openvswitch.org >> Subject: Re: [ovs-dev] [PATCH] netdev-dpdk: Change eth rx burst size. >> >> >> > On 10 Apr 2015, at 00:22, Traynor, Kevin <kevin.tray...@intel.com> wrote: >> > >> > >> >> -----Original Message----- >> >> From: Pravin Shelar [mailto:pshe...@nicira.com] >> >> Sent: Thursday, April 9, 2015 10:50 PM >> >> To: Traynor, Kevin >> >> Cc: Daniele Di Proietto; dev@openvswitch.org >> >> Subject: Re: [ovs-dev] [PATCH] netdev-dpdk: Change eth rx burst size. >> >> >> >> On Thu, Apr 9, 2015 at 2:36 PM, Traynor, Kevin <kevin.tray...@intel.com> >> >> wrote: >> >>> >> >>>> -----Original Message----- >> >>>> From: Pravin Shelar [mailto:pshe...@nicira.com] >> >>>> Sent: Thursday, April 9, 2015 9:10 PM >> >>>> To: Traynor, Kevin; Daniele Di Proietto >> >>>> Cc: dev@openvswitch.org >> >>>> Subject: Re: [ovs-dev] [PATCH] netdev-dpdk: Change eth rx burst size. >> >>>> >> >>>> On Wed, Apr 8, 2015 at 8:43 AM, Kevin Traynor <kevin.tray...@intel.com> >> >>>> wrote: >> >>>>> Change eth rx burst size from 192 to 32. This significantly >> >>>>> improves performance for packets that will be forwarded >> >>>>> through dpdkvhost ports, as the max dpdkvhost tx burst >> >>>>> size (32) will not be exceeded. There are negligible >> >>>>> effects in other scenarios. >> >>>>> >> >>>>> Signed-off-by: Kevin Traynor <kevin.tray...@intel.com> >> >>>> >> >>>> Daniele, >> >>>> You mentioned that you are going to fix the issue by handling >> >>>> different burst size of vhost devices. Do you have the fix? >> >>> >> >>> It wasn't clear in the commit message but just to clarify, the >> >>> dpdkvhost max burst size is set in the DPDK vhost library and if >> >>> we try and enqueue >32 packets only 32 will be enqueued. We can >> >>> add a back-off/retry but it will give better performance to >> >>> ensure that the vhost burst size is not exceeded. >> >>> >> >> >> >> How about adding for loop in vhost_send() to handle burst of packets >> >> larger than 32? >> > >> > Yeah, the DPDK sample app uses a backoff(small sleep)/retry loop but >> > you could put it in a busy loop too. I was planning to add some retry >> > code anyway as it's possible that you could get bursts of >32 from a >> > different interface, so it's good to handle this. >> > >> > The virtqueues are of very limited size (128 packets I think), so they >> > are not ideally suited to bursty traffic which I guess is why DPDK have >> > set the max burst size to 32. Time the PMD spends waiting/retrying is >> > time it could be doing something else, so I think it is better to reduce >> > the burst size where possible. I haven't seen any advantage to the 192 eth >> > rx burst size in my testing as changing to 32 I get negligible difference >> > on phy2phy tests (~1%) and I get a 2x-3x performance boost for vhost >> > loopback tests, but perhaps there are some use cases where it is >> > advantageous? >> >> I agree with you guys that we should use a loop here, >> like in dpdk_queue_flush__(). How much slower would that be? >> >> > > I've tested a simple retry loop with the 192 burst size and the performance > is very poor. With the retry loop and a burst size of 32, I see good > performance. I think this is because with the bigger burst sizes we are > hitting limits/contention in the vhost interface and we have to loop until > they are resolved. > > I think we need a retry loop, but it's not a performant replacement for > optimizing the burst size where we can at the moment. I can share the code > if you want to test it out on your systems. >
We need to loop to handle different burst sizes. So Can you post patch with the loop and change default burst size to 32 for better performance. We can optimize performance for different burst sizes later on. _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
