Some comments below. I looped in Al, Maryam and Billy as they may have some suggestions as to the next steps as they drafted the original draft.
+Al +Maryam +Billy From: Alec Hothan (ahothan) [mailto:ahot...@cisco.com] Sent: Monday, December 19, 2016 4:09 PM To: Gray, Mark D <mark.d.g...@intel.com>; Thomas F Herbert <therb...@redhat.com>; vpp-dev@lists.fd.io Subject: Re: [vpp-dev] vHost user test scenarios for CSIT - TWS meeting Hi Mark, Thanks for providing the pointer to the IETF draft. This looks like a great document for describing vswitch performance benchmarking and we could clearly reuse/contribute to that document. [Gray, Mark D] Yes, I think it would be good to share the good work that you are doing to the wider community and this may be a vehicle to do it. There are already some overlap with what is being discussed here such as packet paths involving 1 or 2 VMs (what we call PVP, PVVP and used as well in OpNFV and EANTC testing), Other packet paths are called V2V, P2P… I think it would be great to make these terms more official in this draft. My main concern is that this document – by design - stays agnostic of the implementations and deployment constraints, more specifically OpenStack base deployment constraints. One example is the number of physical interfaces, as for most vswitch benchmarks, the draft still describes 2 phys interfaces, despite the fact that the vast majority of deployments use 1 phys interface per compite node (or 1 bonded interface). [Gray, Mark D] Absolutely correct. This is a problem with the IETF draft. I understand that this draft can be augmented/amended with new content but it will likely still need to be complemented by another document specifying the specific constraints of an OpenStack deployment, such as: - Number of phys interface - Encapsulation types as described in this thread - Specific VM to vswitch challenges (vhost user) Regarding the metrics, I would also like to see a more formal description on how to measure all these metrics (this is often lacking in standards specifications). Some of the metrics are well covered by current tools (such as RFS 2544 packet loss rate, 0PacketLoss) but many are not supported and are not trivial to measure (especially when the vswitch is to be seen as a black box). I found this gray area to cause quite a bit of confusion in the industry when end users require blanket conformance to some industry standards and NFV vendors need to comply to them… We also discussed in this forum about the need to specify better the traffic sent to the VNFs, such as flow pattern (how many flows, how new flows are created, terminated over time…). I see this is mentioned very quickly in this draft (3.4 flow classification). [Gray, Mark D] Yeah I think more detail in this area would be useful and it looks like you have more specific examples in this area. Anyway, how would you suggest to see this moving forward? Should we try to split this work in 2 parts: - one generic and agnostic of deployment conditions to be done in the IETF draft context (e.g. introduce formal terminologies for packet paths, metrics definitions and measurement procedures, flow definition…) - one more specific to OpenStack documents (perhaps more in the context of OpNFV or OpenStack) [Gray, Mark D] I think I saw them as one document but your suggestion sounds more reasonable as this would need to applicable outside of the context of Openstack and OPNFV. The IETF process will be clearly a lot longer than the OpNFV/OpenStack one so we will have to work this out as well. [Gray, Mark D] Yeah, I think so but perhaps it is a parallel activity. Best Regards, Alec From: <vpp-dev-boun...@lists.fd.io<mailto:vpp-dev-boun...@lists.fd.io>> on behalf of "Gray, Mark D" <mark.d.g...@intel.com<mailto:mark.d.g...@intel.com>> Date: Friday, December 16, 2016 at 2:23 AM To: Thomas F Herbert <therb...@redhat.com<mailto:therb...@redhat.com>>, "vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>" <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>> Subject: Re: [vpp-dev] vHost user test scenarios for CSIT - TWS meeting Hi, I am arriving a little bit late to this topic. I have been looking over the work and it is really comprehensive and evolving very well! Haveyou considered contributing your work to the IETF draft for benchmarking virtual switches - https://tools.ietf.org/html/draft-vsperf-bmwg-vswitch-opnfv-02 ? It may not be a perfect document but I believe it is trying to be agnostic to a test framework and vswitch implementation and could be a good starting point for a documented test plan that could be reapplied elsewhere and help showcase the benefits of VPP. Maciek, I think you helped on some test cases in this document? Any thoughts on this? Mark From: vpp-dev-boun...@lists.fd.io<mailto:vpp-dev-boun...@lists.fd.io> [mailto:vpp-dev-boun...@lists.fd.io] On Behalf Of Thomas F Herbert Sent: Tuesday, December 13, 2016 5:39 PM To: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> Subject: Re: [vpp-dev] vHost user test scenarios for CSIT - TWS meeting On 12/13/2016 07:53 AM, Billy McFall wrote: I consulted with some of my Red Hat colleagues (primarily ODL developers) and they agree that the typical deployment they have seen is with one interface, not two. +1 Billy McFall On Tue, Dec 13, 2016 at 2:27 AM, Pierre Pfister (ppfister) <ppfis...@cisco.com<mailto:ppfis...@cisco.com>> wrote: As per my action item from last meeting, I asked RedHat guy why they came up with testing with 2 physical interfaces instead of just one. And the answer is really that they did that out of simplicity. We should probably run some tests and see if using a single interface makes a difference (and take a decision based on that). The person I talked to also mentioned that it is desirable to run tests with realistic flows and packet size rather than focusing on the very limited 64B Mpps test. - Pierre Le 7 déc. 2016 à 01:20, Maciek Konstantynowicz (mkonstan) <mkons...@cisco.com<mailto:mkons...@cisco.com>> a écrit : REMINDER: Next call details: https://wiki.fd.io/view/TWS CSIT - vHost user test scenarios for CSIT Wednesday, 7 December 2016 09:00 | PST Time | 1 hr 17:00 | GMT Time (London, GMT) | 1 hr Join WebEx meeting: https://cisco.webex.com/ciscosales/j.php?MTID=m2d165f5c9f3fdf722826e7c05a4499c9 Meeting number: 202 237 426 Meeting password: RZrXrBac (79797222 from phones) -Maciek On 2 Dec 2016, at 17:17, Maciek Konstantynowicz (mkonstan) <mkons...@cisco.com<mailto:mkons...@cisco.com>> wrote: Many thanks to all who attended the TWS call today. Notes were taken on #fdio-meeting IRC and CSIT wiki per links below. We agreed to have a follow-up TWS call to finish the live discussion - on Wednesday 7-Dec 09:00-10:00 PST, call details below and on FD.io<http://fd.io/> TWS page. Minutes: * Meeting ended Fri Dec 2 18:07:36 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) * Minutes: http://ircbot.wl.linuxfoundation.org/meetings/fdio-meeting/2016/fdio-meeting.2016-12-02-16.58.html * Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/fdio-meeting/2016/fdio-meeting.2016-12-02-16.58.txt * Log: http://ircbot.wl.linuxfoundation.org/meetings/fdio-meeting/2016/fdio-meeting.2016-12-02-16.58.log.html Updates: https://wiki.fd.io/view/CSIT/vhostuser_test_scenarios Next call details: https://wiki.fd.io/view/TWS CSIT - vHost user test scenarios for CSIT Wednesday, 7 December 2016 09:00 | PST Time | 1 hr 17:00 | GMT Time (London, GMT) | 1 hr Join WebEx meeting: https://cisco.webex.com/ciscosales/j.php?MTID=m2d165f5c9f3fdf722826e7c05a4499c9 Meeting number: 202 237 426 Meeting password: RZrXrBac (79797222 from phones) -Maciek On 1 Dec 2016, at 15:05, Maciek Konstantynowicz (mkonstan) <mkons...@cisco.com<mailto:mkons...@cisco.com>> wrote: Here call details: vHost user test scenarios for CSIT - TWS meeting Friday, 2 December 2016 17:00 | GMT Time (London, GMT) | 1 hr 09:00 | PST Time | 1 hr Join WebEx meeting: https://cisco.webex.com/ciscosales/j.php?MTID=m80cb727098e36de668591ffdcf11ad83 Meeting number: 203 819 086 Meeting password: A335rJNd (23357563 from phones) Also updated here: https://wiki.fd.io/view/TWS -Maciek vHost user test scenarios for CSIT - TWS meeting Friday, 2 December 2016 17:00 | GMT Time (London, GMT) | 1 hr Join WebEx meeting: https://cisco.webex.com/ciscosales/j.php?MTID=m80cb727098e36de668591ffdcf11ad83 Meeting number: 203 819 086 Meeting password: A335rJNd (23357563 from phones) Join by phone +1-408-525-6800<tel:%2B1-408-525-6800> Call-in toll number (US/Canada) +1-866-432-9903<tel:%2B1-866-432-9903> Call-in toll-free number (US/Canada) Access code: 203 819 086 Global call-in numbers | Toll-free calling restrictions On 1 Dec 2016, at 10:56, Maciek Konstantynowicz (mkonstan) <mkons...@cisco.com<mailto:mkons...@cisco.com>> wrote: OK, this sounds convincing at least to em. Set for 09:00 - 10:00am PST tomorrow, Friday, 2-Dec. I will set up webex, and advertise it on csit-dev and vpp-dev and on TWS page. Give me an hour or so.. working in PST TZ makes me slower than UK - is it CA air? ;) -Maciek On 1 Dec 2016, at 05:02, Thomas F Herbert <therb...@redhat.com<mailto:therb...@redhat.com>> wrote: On 11/30/2016 08:41 PM, Maciek Konstantynowicz (mkonstan) wrote: All, We didn’t have much time to discuss the vhostuser use cases on CSIT project call earlier today. On the call we also didn’t have all the interested parties that were providing feedback to date. Hence agreed to hold a separate Tech WorkStream call to cover this. Quick check on #fdio-csit irc, this Friday AM PST seems to work for folks. What about 08:00-09:00 PST this Friday? 0900-10:00 PST would be better for me. If yes, I will post meeting details here: https://wiki.fd.io/view/TWS -Maciek On 29 Nov 2016, at 18:53, Thomas F Herbert <therb...@redhat.com<mailto:therb...@redhat.com>> wrote: Maciek, Thanks! --TFH On 11/29/2016 09:27 PM, Maciek Konstantynowicz (mkonstan) wrote: All, Here is the first draft: https://wiki.fd.io/view/CSIT/vhostuser_test_scenarios I did my best to capture all inputs as per this thread. But it’s hardly readable yet - requires more TLC :) See what you think - feel free to add/edit things directly on FD.io<http://fd.io/> wiki page. Suggest to discuss next steps on csit weekly call tomorrow, details here: https://wiki.fd.io/view/CSIT/Meeting -Maciek On 28 Nov 2016, at 07:37, Thomas F Herbert <therb...@redhat.com<mailto:therb...@redhat.com>> wrote: All, At last week's CSIT meeting, Maciek (mkons...@cisco.com<mailto:mkons...@cisco.com>) offered to compile a summary suggestions on this mailing list. On 11/22/2016 11:34 AM, Pierre Pfister (ppfister) wrote: Hello Thomas, Sorry I haven't reached out faster, I was travelling. Please have a look at vppsb/vhost-test It includes a standalone script which provides VPP and VM configuration for PVP tests. - Runs testpmd in the VM - Supports various CPU configuration for VPP - Can run with or without gdb, debug or release Not committed yet: - Supports VM restart - Support for VPP restart - Support for multiple additional (dead) vhost interface I did that outside of the context of CSIT so people can: - Look at it and see what are the optimisations that are used - Use it without CSIT I will keep using and improving it because I use it for my own development and testing purposes. Rest of this inline. Le 8 nov. 2016 à 22:25, Thomas F Herbert <therb...@redhat.com<mailto:therb...@redhat.com>> a écrit : All: Soliciting opinions from people as to vhost-user testing scenarios and guest modes in fd.io<http://fd.io/> CSIT testing of VPP - vhost-user. I will forward to this mailing list as well as summarize any additional feedback. I asked some people that happen to be here at OVSCON as well as some Red Hat and Intel people. I am also including some people that are involved in upstream vhost-user work in DPDK. So far, I have the following feedback with an attempt to condense feedback and to keep the list small. If I left out anything, let me know. In addition to the PVP tests done now with small packets. Testpmd in guest is OK for now. 1 Add multiple VMs (How many?) Makes sense to me. 2 is enough (4 would be good number). 2 Both multi-queue and single-queue Yes. Ideally, 1-2-4 queues. With different number of workers (0 workers, i.e. single VPP thread, 1 worker, queues*2 workers). 3 Tests that cause the equivalent of multiple flows in OVS. Varying variety of traffic including layer 2 and layer 3 traffic. Yes. Should test with L2 and L3. 4 Multiple IF's (Guest or Host or Both?) Possibly. But more importantly, I think, we need to have VM restart and interface restart (delete - create). OpenStack integration generates a significant amount of delete-recreate of vhost interface. The following might not be doable by 17.01 and if not consider the following as a wish list for future: 1 vxLan tunneled traffic 2 VPP in guest with layer 2 and layer 3 vRouted traffic. 3 Additional Overlay/Underlay: MPLS --TFH -- Thomas F Herbert SDN Group Office of Technology Red Hat _______________________________________________ vpp-dev mailing list vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> https://lists.fd.io/mailman/listinfo/vpp-dev -- Thomas F Herbert SDN Group Office of Technology Red Hat -- Thomas F Herbert SDN Group Office of Technology Red Hat -- Thomas F Herbert SDN Group Office of Technology Red Hat _______________________________________________ vpp-dev mailing list vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> https://lists.fd.io/mailman/listinfo/vpp-dev _______________________________________________ vpp-dev mailing list vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> https://lists.fd.io/mailman/listinfo/vpp-dev -- Thomas F Herbert SDN Group Office of Technology Red Hat
_______________________________________________ vpp-dev mailing list vpp-dev@lists.fd.io https://lists.fd.io/mailman/listinfo/vpp-dev
