On 06/04/2012 04:00 PM, Kevin L. Mitchell wrote:
Today I've noticed some significant problems with nova's test suite
leaving literally hundreds of python processes out. I'm guessing that
this has to do with the unit tests for the multiprocess patch, which was
just approved. This could be causing problems with jenkins, too…
Anybody have any other insights?
Yes, several Jenkins slaves have been taken out by running nova unit
tests. The one that I am able to log into seems to be continuously
respawing python processes. Other slaves are inaccessible due to having
exhausted their RAM.
I note that all of the tests run after that change merged carry this
warning notice from Jenkins:
Process leaked file descriptors. See
http://wiki.jenkins-ci.org/display/JENKINS/Spawning+processes+from+build
for more information
So I think it's fair to say that Jenkins corroborates your suspicion
that change introduced a problem with leaking processes.
This is affecting any Jenkins slave that the nova unit tests job runs
on, which in turn affects jobs for unrelated projects that happen to
later run on that slave.
In addition to correcting this problem, I believe we should add a build
step to Jenkins to ensure that all of the test processes have terminated
correctly.
-Jim
_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
