On 6/22/2015 4:38 PM, Matt Riedemann wrote:
On 6/22/2015 4:32 PM, Russell Bryant wrote:
On 06/22/2015 05:23 PM, Matt Riedemann wrote:
The check-tempest-dsvm-cells job has been in nova's check queue since
January as non-voting and has been stable for a couple of weeks now, so
before it's regressed melwitt proposed a change to making it voting and
gating on nova changes [1].
I raised a concern in that change that the tempest-dsvm-cells job is not
in the check queue for tempest or devstack changes, so if a change is
merged in tempest/devstack which breaks the cells job, it will block
nova changes from merging.
mtreinish noted that tempest already has around 30 jobs running against
it right now in the check queue, so he'd prefer that another one isn't
added since the nova API shouldn't be different in the case of cells,
but we know there are quirks. That can be seen from the massive regex
of excluded tests for the tempest-dvsm-cells job [2].
If we could turn that regex list into tempest configurations, I think
that would make it possible to not have to run tempest changes through
the cells job in the check queue but also feel reasonably confident that
changes to tempest that use the config options properly won't break the
cells job (and block nova in the gate).
This is actually something we should do regardless of voting or not on
nova since new tests get added which might not fall in the regex and
break the cells job. So I'm going to propose some changes so that the
regex will be moved to devstack-gate (regex exodus (tm)) and we'll work
on whittling down the regex there (and run those d-g changes against the
tempest-dsvm-cells job in the experimental queue).
The question for the nova team is, shall we make the tempest-dsvm-cells
job voting on nova changes knowing that the gate can be broken with a
change to tempest that isn't caught in the regex? In my opinion I think
we should make it voting so we don't regress cells with changes to nova
that go unnoticed with the non-voting job today. Cells v2 is a nova
priority for Liberty so we don't want setbacks now that it's stable.
If a change does land in tempest which breaks the cells job and blocks
nova, we (1) fix it or (2) modify the regex so it's excluded until fixed
as has been done up to this point.
I think we should probably mull this over in the ML and then vote on it
in this week's nova meeting.
[1] https://review.openstack.org/#/c/190894/
[2]
http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/devstack-gate.yaml#n1004
Regarding your "regex exodus", I recently added something for this. In
another project, I'm setting the regex in a file I keep in the code repo
instead of project-config.
support for DEVSTACK_GATE_SETTINGS in devstack-gate:
https://review.openstack.org/190321
usage in a job definition: https://review.openstack.org/190325
a DEVSTACK_GATE_SETTINGS file that sets DEVSTACK_GATE_TEMPEST_REGEX:
https://review.openstack.org/186894
It all seems to be working for me, except I still need to tweak my regex
to get the job passing, but at least I can do that without updating
project-config now.
Awesome, that is way cleaner. I'll go that route instead, thanks!
Here is the change that moves the regex into nova:
https://review.openstack.org/#/c/194411/
--
Thanks,
Matt Riedemann
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
