Karl, Thanks for all answers and clarifications! ~23Mpps that’s indeed your NIC - we’re observing same in CSIT. One more point re host setup - I read it’s CPU power management disabled, CPU TurboBoost disabled, and all memory channels populated? I looks so, but wanted to recheck as not listed on your opening slide :)
-Maciek > On 15 Feb 2017, at 21:55, Karl Rister <kris...@redhat.com> wrote: > > On 02/15/2017 03:28 PM, Maciek Konstantynowicz (mkonstan) wrote: >> Thomas, many thanks for sending this. >> >> Few comments and questions after reading the slides: >> >> 1. s3 clarification - host and data plane thread setup - vswitch pmd >> (data plane) thread placement >> a. "1PMD/core (4 core)” - HT (SMT) disabled, 4 phy cores used for >> vswitch, each with data plane thread. >> b. “2PMD/core (2 core)” - HT (SMT) enabled, 2 phy cores, 4 logical >> cores used for vswitch, each with data plane thread. >> c. in both cases each data plane thread handling a single interface >> - 2* physical, 2* vhost => 4 threads, all busy. > > Correct. > >> d. in both cases frames are dropped by vswitch or in vring due to >> vswitch not keeping up - IOW testpmd in kvm guest is not DUT. > > That is the intent. > >> 2. s3 question - vswitch setup - it is unclear what is the forwarding >> mode of each vswitch, as only srcIp changed in flows >> a. flow or MAC learning mode? > > In OVS we program flow rules that pass bidirectional traffic between a > physical and vhost port pair. > >> b. port to port crossconnect? > > In VPP we are using xconnect. > >> 3. s3 comment - host and data plane thread setup >> a. “2PMD/core (2 core)” case - thread placement may yield different >> results >> - physical interface threads as siblings vs. >> - physical and virtual interface threads as siblings. > > In both OVS and VPP a physical interface thread is paired with a virtual > interface thread on the same core. > >> b. "1PMD/core (4 core)” - one would expect these to be much higher >> than “2PMD/core (2 core)” >> - speculation: possibly due to "instruction load" imbalance >> between threads. >> - two types of thread with different "instruction load": >> phy->vhost vs. vhost->phy >> - "instruction load" = instr/pkt, instr/cycle (IPC efficiency). >> 4. s4 comment - results look as expected for vpp >> 5. s5 question - unclear why throughput doubled >> a. e.g. for vpp from "11.16 Mpps" to "22.03 Mpps" >> b. if only queues increased, and cpu resources did not, or have they? >> 6. s6 question - similar to point 5. - unclear cpu and thread reasources. > > Queues and cores increase together. In the host single queue used 4 PMD > threads on 2 core, two queue uses 8 PMD threads on 4 cores, and three > queue uses 12 PMD threads on 6 cores. In the guest we used 2, 4, and 6 > cores in testpmd without using sibling hyperthreads in order to avoid > bottlenecks in the guest. > >> 7. s7 comment - anomaly for 3q (virtio multi-queue) for (srcMAc,dstMAC) >> a. could be due to flow hashing inefficiency. > > That was our thinking and where we were going to look first. > > I think I have tracked the three queue issue to using too many mbufs for > multi-queue as addressed in this commit: > > https://github.com/vpp-dev/vpp/commit/a06dfb39c6bee3fbfd702c10e1e1416b98e65455 > > I originally used the suggestion of 131072 from this page: > > https://wiki.fd.io/view/VPP/Command-line_Arguments#.22dpdk.22_parameters > > I'm now testing with 3 queue and 32768 mbufs and getting in excess of 23 > Mpps across all the flow configurations except the one with the hashing > issue. For our hardware configuration we believe this is hardware > limited and could potentially go faster (as mentioned on slide 6). > >> >> -Maciek >> >>> On 15 Feb 2017, at 17:34, Thomas F Herbert <therb...@redhat.com >>> <mailto:therb...@redhat.com>> wrote: >>> >>> Here are test results on VPP 17.01 compared with OVS/DPDK 2.6/1611 >>> performed by Karl Rister of Red Hat. >>> >>> This is PVP testing with 1, 2 and 3 queues. It is an interesting >>> comparison with the CSIT results. Of particular interest is the drop >>> off on the 3 queue results. >>> >>> --TFH >>> >>> >>> -- >>> *Thomas F Herbert* >>> SDN Group >>> Office of Technology >>> *Red Hat* >>> <vpp-17.01_vs_ovs-2.6.pdf>_______________________________________________ >>> vpp-dev mailing list >>> vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io> >>> https://lists.fd.io/mailman/listinfo/vpp-dev >> > > > -- > Karl Rister <kris...@redhat.com> _______________________________________________ vpp-dev mailing list vpp-dev@lists.fd.io https://lists.fd.io/mailman/listinfo/vpp-dev
