We actually collected latency data on all of the test runs that we performed. There are a couple of sticking points however.
1) Our current packet generator (MoonGen+lua-trafficgen [1]) relies on the Intel 82599ES PTP hardware in order to measure packet latency. In our testing that works very well up to a certain point (somewhere in the neighborhood of 9.3 Mpps per port) and then it starts to fall apart with fewer and fewer latency packets being successfully sent out and even fewer being properly processed on the receive side. Since our testing involved multi-queue and VPP scaled so well in many of the different flow configurations we feel our latency data is incomplete -- for lower performing configurations we have nice, complete sets of data but for higher performing configurations (the majority) the data sets are far from complete. I believe the trends in the data collected are fairly representative, but I would prefer to do a real comparison where this problem does not exist. This is one reason that we are very interested in using TRex and/or Intel XL710 adapters. 2) We feel that latency comparisons are best made at like throughput levels. If we were truly going to do a latency comparison between two stacks (in this case OVS and VPP) we would most likely pick a fixed packet rate. Karl [1] https://github.com/atheurer/lua-trafficgen On 02/27/2017 05:34 PM, Zhou, Danny wrote: > Tom, more comments inline. > > > > *From:*Thomas F Herbert [mailto:therb...@redhat.com] > *Sent:* Tuesday, February 28, 2017 12:41 AM > *To:* Zhou, Danny <danny.z...@intel.com>; vpp-dev@lists.fd.io > *Cc:* Karl Rister <kris...@redhat.com>; csit-...@lists.fd.io > *Subject:* Re: [vpp-dev] FW: Re: Interesting perf test results from Red > Hat's test team > > > > Hi Danny, > > My response is inline below. > > > > On 02/27/2017 03:01 AM, Zhou, Danny wrote: > > Hi Tom, Karl, > > > > In the last “Todo List” page, there is RFC2544 0% packet loss test > item, do you want to measure the min/max latency as well? > > > Is your question particularly about vhost-user testing? > Also, I am interested in why you brought up latency. Are you doing your > own latency testing in conjunction with your nsh-sfc work and do you > think we need to do latency in CSIT as well in testing nsh-sfc perf on > vhosts? > > /[Zhou, Danny] No, I am not particularly interested in latency about > vhost_user. Instead, as you mentioned I am interested in VPP as well as > NSH_SFC latency //J//. Because during our internal P2P NSH_SFC RFC2544 > test without vhost, we observed/*//*/maximum latency is about 100x more > than minimum latency, which looks like critical issue for some use cases > that requires real time environment. I would like to know if VPP or DPDK > has the similar characteristics, or just because I did not use real time > kernel as you guys did. Anyway, look forward your results, in the > meantime will share our results once we figure out root cause./ > > > I will let Karl answer as to the testing he has done and is doing. > However, in fd.io CSIT, the latency for vhost-user results are here: > https://docs.fd.io/csit/rls1701/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.html > > --TFH > > > > -Danny > > > > *From:*vpp-dev-boun...@lists.fd.io > <mailto:vpp-dev-boun...@lists.fd.io> > [mailto:vpp-dev-boun...@lists.fd.io] *On Behalf Of *Thomas F Herbert > *Sent:* Friday, February 17, 2017 1:55 AM > *To:* vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io> > *Cc:* Karl Rister <kris...@redhat.com> <mailto:kris...@redhat.com> > *Subject:* Re: [vpp-dev] FW: Re: Interesting perf test results from > Red Hat's test team > > > > > > Jan, > I have answered below but am forwarding this to Karl who performed > the testing to get the exact answers. > > --TFH > > On 02/16/2017 08:59 AM, Jan Scheurich wrote: > > ____________________________________________ > *From:* Jan Scheurich > *Sent:* Thursday, 16 February, 2017 14:41 > *To:* therb...@redhat.com <mailto:therb...@redhat.com> > *Cc:* vpp-...@fd.io <mailto:vpp-...@fd.io> > *Subject:* Re: [vpp-dev] Interesting perf test results from Red > Hat's test team > > > > > > Hi Thomas, > > > > Thanks for these interesting measurements. I am not quite sure I > fully understand the different configurations and traffic cases > you have been testing: > > > > · */[Zhou, Danny] /*ToDo you vary the number of > vhost-user queues to the guest and/or the number of RX queues > for the phy port? > > These are vhost user queues. Because OVS and/or VPP is running in > the host. > > > · Did you add cores at the same time you added queues? > > Yes > > > · When you say flows, do you mean L3/L4 packet flows > (5-tuples) or forwarding rules/flow rules? > > These are L2 matches. > > > · When you e.g. say N flows (srcip, dstip) do you mean > matching on these fields, modifying these fields or both > > > > Would it be possible to provide the exact VPP and OVS > configurations that were used (ports, queues, cores, ports, > forwarding rules/flows)? > > > > Thanks, Jan > > > > > > Here are test results on VPP 17.01 compared with OVS/DPDK 2.6/1611 > > performed by Karl Rister of Red Hat. > > > > This is PVP testing with 1, 2 and 3 queues. It is an interesting > > comparison with the CSIT results. Of particular interest is the > drop off > > on the 3 queue results. > > > > --TFH > > > > > > -- > > *Thomas F Herbert* > > SDN Group > > Office of Technology > > *Red Hat* > > -------------- next part -------------- > > An HTML attachment was scrubbed... > > URL: > > <_http://lists.fd.io/pipermail/vpp-dev/attachments/20170215/83249b21/attachment-0001.html_> > > -------------- next part -------------- > > A non-text attachment was scrubbed... > > Name: vpp-17.01_vs_ovs-2.6.pdf > > Type: application/pdf > > Size: 243918 bytes > > Desc: not available > > URL: > > <_http://lists.fd.io/pipermail/vpp-dev/attachments/20170215/83249b21/attachment-0001.pdf_> > > > > > > > > > > > > _______________________________________________ > > vpp-dev mailing list > > vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io> > > https://lists.fd.io/mailman/listinfo/vpp-dev > > > > -- > *Thomas F Herbert* > SDN Group > Office of Technology > *Red Hat* > > > > -- > *Thomas F Herbert* > SDN Group > Office of Technology > *Red Hat* > -- Karl Rister <kris...@redhat.com> _______________________________________________ vpp-dev mailing list vpp-dev@lists.fd.io https://lists.fd.io/mailman/listinfo/vpp-dev
