On Thu, Jan 16, 2014 at 6:28 AM, John Griffith <john.griff...@solidfire.com> wrote: > Hey Everyone, > > A while back I started talking about this idea of requiring Cinder > driver contributors to run a super simple cert script (some info here: > [1]). Since then I've been playing with introduction of a third party > gate check here in my own lab. My proposal was to have a non-voting > check that basically duplicates the base devstack gate test in my lab, > but uses different back-end devices that I have available configured > in Cinder to run periodic tests against. Long term I'd like to be > able to purpose this gear to also do something "more useful" for the > over all OpenStack gating effort but to start it's strictly an > automated verification of my Cinder driver/backend. > > What I'm questioning is how to report this information and the > results. Currently patches and reviews are our mechanism for > triggering tests and providing feedback. Myself and many other > vendors that might like to participate in something like this > obviously don't have the infrastructure to try and run something like > this on every single commit. Also since it would be non-voting it's > difficult to capture and track the results. > > One idea that I had was to set something like what I've described > above to run locally on a periodic basis (weekly, nightly etc) and > publish results to something like a "third party verification > dashboard". So the idea would be that results from various third > party tests would all adhere to a certain set of criteria WRT what > they do and what they report and those results would be logged and > tracked publicly for anybody in the OpenStack community to access and > view?
My concern here is how to identify what patch broke the third party thing. If you run this once a week, then there are possible hundreds of patches which might be responsible. How do you identify which one is the winner? > Does this seem like something that others would be interested in > participating in? I think it's extremely valuable for projects like > Cinder that have dozens of backend devices, and regardless of other > interest or participation in the community I intend to implement > something like this on my own regardless. It would just be > interesting to see if we could have an organized and official effort > to gather this sort of information and run these types of tests. > > Open to suggestions and thoughts as well as any of you that may > already be doing this sort of thing. By the way, I've been looking at > things like SmokeStack and other third party gating checks to get some > ideas as well. Michael -- Rackspace Australia _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
