I agree on the issue with the output from generated tests. That is troublesome, but from what I've seen in the source code, probably something that could be remedied. It's also very generous in it's parallel execution which is fine client-side, but can overwhelm a test environment since there's no configuration to throttle back the number of tests being executed at a time. Unfortunately I haven't seen a Python test runner that meets all the criteria that I'd like to have, thus this and other little proof of concepts I've been tossing around to see if any better approaches are out there.
Daryl ________________________________________ From: openstack-qa-team-bounces+daryl.walleck=rackspace....@lists.launchpad.net [openstack-qa-team-bounces+daryl.walleck=rackspace....@lists.launchpad.net] on behalf of Jaroslav Henner [jhen...@redhat.com] Sent: Monday, September 24, 2012 7:28 AM To: openstack-qa-team@lists.launchpad.net Subject: [Openstack-qa-team] PyVows proof of concept In reply to: https://lists.launchpad.net/openstack-qa-team/msg00236.html, which didn't came to my mailbox for some reason (attachment?) I tried pyVows myself. I kinda liked the concept, but I didn't like the way it is reporting to JUnit format XML when using "generative testing": http://heynemann.github.com/pyvows/#-using-generative-testing In Jenkins, it looked like: Test Result : Add ----------------- should_be_numeric 0 ms Passed should_be_numeric 0 ms Passed should_be_numeric 0 ms Passed should_be_numeric 0 ms Passed should_be_numeric 0 ms Passed should_be_numeric 0 ms Passed The parameters to the testing method are important when using generative testing, so I think they should be included in the name of the test. But some funny characters like ()%* I don't remember which are causing problems in Jenkins. I was investigating some problems with them months ago with some other testing framework. I don't know how to address this problem. It may be worthy to consider making some Robot framework outputs plugin if generative testing is needed, or use Robot Framework https://wiki.jenkins-ci.org/display/JENKINS/Robot+Framework+Plugin J.H. -- Mailing list: https://launchpad.net/~openstack-qa-team Post to : openstack-qa-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack-qa-team More help : https://help.launchpad.net/ListHelp -- Mailing list: https://launchpad.net/~openstack-qa-team Post to : openstack-qa-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack-qa-team More help : https://help.launchpad.net/ListHelp
