On 07/09/2013 11:24 AM, Joshua McKenty wrote: > Monty, you missed a couple of elements of the original program > definition - the mappings from the tempest based "scorecard" to specific > versions of the API docs, and the availability of the tested and > verified endpoints to "tools ecosystem" developers.
Yes, great catch. > Also, I believe the definition of "compliance" rests with the Board and > not the TC, despite some debate as to whether we're capable of handling it. I think it's a collaboration. I think the board owns the definition of compliance, but I think that the TC has got to own the technical implementation of that definition, both in the tooling but also in the technical assessment of the results of tests. I don't think we need to get legalistic with this yet... let's get something in front of people, then we can battle on who gets to own looking at test result matrixes. :) > And I'll arm-wrestle you for the PTL slot ;) We should do that on a web-case... > PS - for the purposes of FITS, the "candidate cloud" can be either a > public cloud, or a test environment for private cloud software. Yes, I believe someone would need to be able to run a copy of this locally as well. > On Jul 9, 2013 8:03 AM, "Monty Taylor" <mord...@inaugust.com > <mailto:mord...@inaugust.com>> wrote: > > Hey all, > > I'd like to propose an official program to the TC - refstack, a program > for verifying interoperability between implementations via FITS testing. > > Official Title: OpenStack Interoperability > Initial PTL: Monty Taylor <mord...@inaugust.com > <mailto:mord...@inaugust.com>> > Mission Statement: > Develop and maintain FITS testing and interoperability reporting for > OpenStack deployments and distributions. > > For a little background, the specific incarnation of the idea for this > came up at the last in-person Foundation Board meeting, and I think it's > a good idea. Also, it turns out there’s a reference in the logo > guidelines stating that any FITS defined by the TC and made available by > OpenStack needs to be passed before the logos can be used for a product. > > Follow on discussions were held at the summit, and a general approach of > basing the testing of clouds on tempest was agreed to. The general idea > is that for any given cloud, refstack will run tempest against the cloud > in a standard configuration (not re-configuring tempest on a per-cloud > basis) This will result in a set of passing and failing tests. That > output then needs to be processed and presented in a way that it can be > the basis of determining which elements are compliant and not. The > ultimate decisions around which elements would be 'required' to be a > compliant deployment would come back to the TC, and how that compliance > translates in to trademark usage goes back to the board. But it's the > job of refstack to be able to test the candidate cloud and report on its > actual capabilities. > > In conjunction with this a 'reference' deployment config is needed, so > that we can validate that our test is passable. At the moment I would > expect this to be the de-facto standard which is devstack, but that's > ultimately a happenstance and opportunistic config, and over time the > refstack program would be involved in helping to define a deployment > configuration or configurations that we as a community feel should > reasonably be expected to pass our own FITS testing. > > Expected outputs: > - A service for submission of cloud endpoint information for testing > - Processing of tempest output into a service compliance report > - One or more deployment configurations that can, themselves, pass the > testing 100% > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > <mailto:OpenStack-dev@lists.openstack.org> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
