On 13.9.2012 18:21, Jay Pipes wrote:
On 09/13/2012 06:43 AM, Sean Dague wrote:
On 09/13/2012 12:42 AM, Vishvananda Ishaya wrote:
Brian Waldon and I discussed this today. We came up with an idea which is to have the
tempest run on stable/<last-release> during the dev cycle, and after the last
milestone turn it on during the bugfixing phase. The potential disadvantage of this
approach is that it could potentially make bugfixes a little harder to get in, but it
might help make things more stable.
Definitely worth a discussion at the summit.
I think that would be a great idea. I'm still not convinced we need to
fully give up on tempest running fully on normal commits. Post Folsom
release / pre summit, I'm going to take some time to dive into the
problem. The nose auto parallelizing isn't currently working, but some
manual approaches might.
I think unittests may be relatively easily split between several Jenkins
jobs just by specifying which packages to run. Following might be
executed in different Jobs:
nose -w PATH_TO_TEMPEST tempest.tests.compute
nose -w PATH_TO_TEMPEST tempest.tests.image
nose -w PATH_TO_TEMPEST tempest.tests.network
The nose parallel issue:
https://github.com/nose-devs/nose/issues/550
and another blocking issue:
https://github.com/nose-devs/nose/issues/551
Frankly, I am seriously doubting whether nosetests is the right solution
to our problems. I know nosetests has a lot of features but there are
serious impedance mismatches between nosetests (and the underlying
unittest[2] library) that make it unwieldy for full functional
integration tests.
Unfortunately, it is a serious investment in time to switch to something
else, considering the whole Tempest codebase pretty much depends on
nose/unittest.
We are at the point, however, where we really do need to solve the
parallelization problem -- the more tests we add, the longer tempest
takes to run, and the longer tempest takes, the smaller the chance of
the full run being the gate for core projects.
Have you guys considered DTests? https://github.com/klmitch/dtest
There is also Jenkins plugin that can distribute JUnit tests between
several workers. It would be great to have something same for Python
unittest/nose/whatever. Surely there would have to be some thinking done
about how to handle class/module fixtures.
Personally, I'm torn between spending time on a rewrite and trying to
decipher nose's often incomprehensible codebase and pushing fixes to the
upstream library.
-jay
--
Mailing list: https://launchpad.net/~openstack-qa-team
Post to : openstack-qa-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack-qa-team
More help : https://help.launchpad.net/ListHelp
