I'm certainly all for anything that makes things easier. However, I do want to make sure that if we migrate runners, we should make sure that the new implementation solves all the issues we're trying to address.
Daryl ________________________________________ From: openstack-qa-team-bounces+daryl.walleck=rackspace....@lists.launchpad.net [openstack-qa-team-bounces+daryl.walleck=rackspace....@lists.launchpad.net] on behalf of David Kranz [david.kr...@qrclab.com] Sent: Thursday, September 27, 2012 8:11 PM To: openstack-qa-team@lists.launchpad.net Subject: Re: [Openstack-qa-team] PyVows proof of concept We discussed this a bit at the meeting today. Monty has proposed a session on the QA track about parallelizing some of the CI stuff. He believes tempest could share the parallelization code. See http://summit.openstack.org/cfp/details/69. Parallelizing the tempest gate job is as much of a ci issue as a tempest issue and working with them, and their proposal, could make things much easier for us IMO. -David On 9/27/2012 8:10 PM, Daryl Walleck wrote: > I agree on the issue with the output from generated tests. That is > troublesome, but from what I've seen in the source code, probably something > that could be remedied. It's also very generous in it's parallel execution > which is fine client-side, but can overwhelm a test environment since there's > no configuration to throttle back the number of tests being executed at a > time. Unfortunately I haven't seen a Python test runner that meets all the > criteria that I'd like to have, thus this and other little proof of concepts > I've been tossing around to see if any better approaches are out there. > > Daryl > ________________________________________ > From: > openstack-qa-team-bounces+daryl.walleck=rackspace....@lists.launchpad.net > [openstack-qa-team-bounces+daryl.walleck=rackspace....@lists.launchpad.net] > on behalf of Jaroslav Henner [jhen...@redhat.com] > Sent: Monday, September 24, 2012 7:28 AM > To: openstack-qa-team@lists.launchpad.net > Subject: [Openstack-qa-team] PyVows proof of concept > > In reply to: > https://lists.launchpad.net/openstack-qa-team/msg00236.html, which > didn't came to my mailbox for some reason (attachment?) > > I tried pyVows myself. I kinda liked the concept, but I didn't like the > way it is reporting to JUnit format XML when using "generative testing": > http://heynemann.github.com/pyvows/#-using-generative-testing > > In Jenkins, it looked like: > > Test Result : Add > ----------------- > should_be_numeric 0 ms Passed > should_be_numeric 0 ms Passed > should_be_numeric 0 ms Passed > should_be_numeric 0 ms Passed > should_be_numeric 0 ms Passed > should_be_numeric 0 ms Passed > > The parameters to the testing method are important when using generative > testing, so I think they should be included in the name of the test. But > some funny characters like > ()%* I don't remember which > are causing problems in Jenkins. I was investigating some problems with > them months ago with some other testing framework. I don't know how to > address this problem. It may be worthy to consider making some Robot > framework outputs plugin if generative testing is needed, or use Robot > Framework > > https://wiki.jenkins-ci.org/display/JENKINS/Robot+Framework+Plugin > > J.H. > > -- > Mailing list: https://launchpad.net/~openstack-qa-team > Post to : openstack-qa-team@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack-qa-team > More help : https://help.launchpad.net/ListHelp > -- Mailing list: https://launchpad.net/~openstack-qa-team Post to : openstack-qa-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack-qa-team More help : https://help.launchpad.net/ListHelp -- Mailing list: https://launchpad.net/~openstack-qa-team Post to : openstack-qa-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack-qa-team More help : https://help.launchpad.net/ListHelp
