On 7/14/2016 4:40 PM, Matt Riedemann wrote:
On 7/14/2016 3:11 AM, GHANSHYAM MANN wrote:
1. Always add a schema change to Tempest if a microversion changes a
response.
The problem with this is we shouldn't land a schema change by itself
in tempest.
Until we have something using the schema we have no verification that
they
actually work. We can and will land incorrect schemas if we did this.
That's why
there is a pretty strong policy of only landing code that is run in
CI somewhere
for Tempest.
+1, yes we should not add those without testing.
OK, good point on not landing changes that aren't tested. That's pretty
obvious.
For something like this though:
https://review.openstack.org/#/c/339559/
The gate-tempest-dsvm-neutron-full-ssh is testing it indirectly, so I'm
assuming that's OK even though we don't have an explicit test for the
2.3 microversion?
I know the patch needs to be updated for the other extended server
attributes in that microversion, but it's the immediate thing I want to
get fixed so we can get on with making the
gate-tempest-dsvm-neutron-full-ssh job voting.
We talked about this topic at the nova midcycle and these are the
notes/decisions I took:
* We can't have schema changes in Tempest that aren't tested - this is
already the Tempest policy and makes sense.
* When adding tests for a new microversion, if there is a gap in Tempest
response schema validation it should be filled in that patch.
* After feature freeze we should fill any gaps between nova's latest
microversion and the schema coverage in Tempest. Right now we have a
backlog so we're playing catch up, but that shouldn't happen once we get
caught up.
* It's fine to have microversion tests in Tempest even if they only hit
the nova API/DB because we want the response schema validation (and to
avoid these gaps in coverage).
So I think we can move forward on filling the gap in Tempest (there are
several open changes for review).
I'll also push a docs change to nova [1] to mention that a Tempest test
needs to be added for any microversion which changes the response schema.
[1]
http://docs.openstack.org/developer/nova/code-review.html#microversion-api
--
Thanks,
Matt Riedemann
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
