Chris, Good to hear from you. Sure use case based testing looks good for Release C. But needs further review for a Multi-Release efforts. See my comments in-line. Thanks Prakash Prakash Ramchandran [logo_huawei] R&D USA FutureWei Technologies, Inc Email: [email protected]<mailto:[email protected]> Work: +1 (408) 330-5489 Mobile: +1 (408) 406-5810 2330 Central Expy, Santa Clara, CA 95050, USA
From: Christopher Price [mailto:[email protected]] Sent: Friday, September 02, 2016 11:09 AM To: Prakash Ramchandran; [email protected] Subject: Re: [opnfv-tech-discuss] FW: [Dovetail] Need concept of Capability and Bundles Hi Prakash, Apologies for not being able to attend the entire meeting today. I’d like to raise concerns at this, some of the things below I find confusing. Are you proposing that: - We use the term ”capability” instead of use case? Why? For Release C use case may suffice for Dovetail testing ... But Use Case refers to one capability based on a given Scenario we use to configure build and test. However a platform may need to support both IPv6 and L3 VPN or say SFC capability. Thus we may have to run three test harness or suite (which I term bundle) to verify and validate all capability for a give release with that combination Platform and test suits. - We use the term “bundle” instead of test harness or test tool? Why? Sure we can use test harness or suite. Tool is separate like test runner using those test harness for the said release testing for compliance etc. This seems to increase confusion without a purpose I am able to fathom. Why would we not use regular terms for these things? I would strongly urge that we use normal testing terminology in DoveTail, maybe we can discuss it again next week as I am a little confused as to the motivation for this deviation. Well we had similar confusion on Scenario and these are evolving and we need to think through, I am not claiming this is final, let's go through workflow and also review as how self test, third-party tests will be enabled based on dependencies and what would be better terms to use. It's an open discussion, the motivation is simple how do we help Dovetail test planning, along with tying efforts to provide test plan and results in a consistent way across releases and use cases for different builds and Deployment configurations. I suggest we revisit the other test suite terminology discussion also. The areas listed do not align with any of the agreements/proposals we made at the hackfest, or in the week since. We have yet to identify the use cases we want to address and we are otherwise busy creating names like “defrel” and “rcup” for suites and tools that do not exist and areas as of yet undefined. Test suite terminology is open for debate and not closed yet. I did present the proposal and was interrupted and agreement/alignment can be made after everyone evaluates and have sent this to opnfv-tech-list earlier and due to Release C work most of committers/contributors are still trying to close, so lets work through one more pass next week. Maybe we could also spend a little time updating the IPv6 spec that has been untouched since August 12th pending some promised input. I did that and send the updates by email on Aug 25th (refer my email to you and Dave Neary) as I could not update the files in gerrit. The specs work sure will address, the diagram and write up is on https://etherpad.opnfv.org/pipv6.(use Fontype Monospace to review the diagram) Regards, Chris From: <[email protected]> on behalf of Prakash Ramchandran <[email protected]> Date: Friday 2 September 2016 at 17:29 To: TECH-DISCUSS OPNFV <[email protected]> Subject: [opnfv-tech-discuss] FW: [Dovetail] Need concept of Capability and Bundles Hi all, Here is the details of Dovetail tools building. Please review and we discuss this next week. We need to add 1. Capability - Defines a specific use case to test based on one or more scenarios/APIs/Features eg. IPv6, L3VPN, SFC etc. 2. Bundle - We refer to test harness that needs to test the Capability defined by one or more use cases as part of Capability Now the Pass fail criterions can be defined by Dovetail and report sent to C&C based on its requirements for Compliances under Compliance Verification Program (CVP) But the scoring by Capability and Matrix plus reporting must be defined by C&C for Dovetail to implement as that is out of scope for Dovetail team. Openstack By Module(s) OPNFV By Use Case(s) DefCore Eg. Keystone, Glance, Nova, Neutron, Cinder, Swift DefRel Eg. say IPv6, L3VPN, SFC in Colorado RefStack HP, Rackspace CI/CD nodes (800-1000 nodes) RefRel LF, Community and Third party labs like NH Tempest Non-Admin API based by Modules Dovetail TCUP Docker with micro-kernel versioned copy of tempest to run the RefStack software package locally on test clouds for certification RCUP Docker with micro-kernel versioned copy of Dovetail release test Bundle(harness) for defined Capability. The suggested terms are new and may be modified like some suggest that RefRel be revised as RefRelStack. We need to define proper work flow and assign proper meanings while we continue to progress with some vagueness like we did in case of Scenario in Release B. Plus API testing can progress once we have scoped API from each of the OPNFV Scenario/Feature projects as what is exposed and what is not and that we leave it to MANO WG to deal with. Note the use case approach is a black box approach and as we evolve to Release D,E we will have better definitions of APIs and at that time we can raise the bar for compliance, for now a low bar for CVP is better than no bar, as we jump the hoop. This is multi-release effort and we are just trying to jump start this with industry experiences to help us build it ground up or call it native to OPNFV. Thanks Prakash Prakash Ramchandran [ogo_huawei] R&D USA FutureWei Technologies, Inc Email: [email protected]<mailto:[email protected]> Work: +1 (408) 330-5489 Mobile: +1 (408) 406-5810 2330 Central Expy, Santa Clara, CA 95050, USA From: Prakash Ramchandran Sent: Friday, September 02, 2016 7:56 AM To: 'Tetsuya Nakamura' Subject: FW: [opnfv-tech-discuss] [MANO_WG] Need concept of Capability and Bundles Prakash Ramchandran [ogo_huawei] R&D USA FutureWei Technologies, Inc Email: [email protected]<mailto:[email protected]> Work: +1 (408) 330-5489 Mobile: +1 (408) 406-5810 2330 Central Expy, Santa Clara, CA 95050, USA From: Prakash Ramchandran Sent: Tuesday, August 30, 2016 7:50 PM To: '[email protected]' Subject: [opnfv-tech-discuss] [MANO_WG] Need concept of Capability and Bundles Hi all, Based on the concept moving up the MANO, we will need to add terms Capability and Bundles to address the increased Complexity. Frank introduced Scenario to us we need now to define Capabilities based on set of [Use case, Features, APIs and Scenario] and test Bundles (harness is alternate term) to verify and validate SLA's. Please review and send your feedback. We can go over this in technical meeting Friday if there is slot to discuss this. Also I am including earlier presentation had made at Hackfest for validating comparison with OpenStack and three new terms came to be coined DefRel, RefRel and TCUP corresponding to DefCore, RefStack and TCUP. There may be some questions on this but once we define Capability & Bundles, and try define Matrix for SLA's we are bound to reach for some terms similar or may be use this. All ideas are Open as we move to Release D as we will need to define how we are containing the complexity to test and integrate in 2017. Thanks Prakash Prakash Ramchandran [ogo_huawei] R&D USA FutureWei Technologies, Inc Email: [email protected]<mailto:[email protected]> Work: +1 (408) 330-5489 Mobile: +1 (408) 406-5810 2330 Central Expy, Santa Clara, CA 95050, USA _______________________________________________ opnfv-tech-discuss mailing list [email protected] https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
_______________________________________________ opnfv-tech-discuss mailing list [email protected] https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
