On 10/18/2013 12:04 PM, Brant Knudson wrote:
To provide a bit more background... Keystone has a bunch of
keystoneclient tests for the v2 API. These tests actually "git clone" a
version of keystoneclient (master, essex-3, and 0.1.1)[0], to use for
testing. Maybe at some point the tests were just for client-server
compatibility, but now they're used for more than that; for example,
they're used for tests that require going through the paste pipeline and
v2 controller. It's just the quickest way to get some tests written. In
addition, there's versions of the client tests for both the kvs and sql
backends.
This causes several problems,
1) It looks like we're not keeping the versions to test up-to-date --
should we checkout supported releases instead?
2) "git clone"ing the keystoneclient doesn't work well with parallel
testing (we have a similar problem in our tests with our "pristine"
database backup)
Can you go into the specifics of why?
3) These tests eat up lots of memory which we've gotten complaints about.
Again, can you go into the specifics of why?
Getting v3 API keystoneclient/keystone testing in tempest is going to
hopefully lead to getting the v2 tests out of Keystone. Thanks to Steve
for taking this first step!
For the v3 API, the tests don't use the keystoneclient but instead use
webtest [1] and the REST API.
[0]
https://github.com/openstack/keystone/blob/master/keystone/tests/test_keystoneclient.py#L1070
[1]
https://github.com/openstack/keystone/blob/master/keystone/tests/test_content_types.py#L69
So v3 keystone API is one thing, but I'm a little concerned with moving
the client testing to Tempest haphazardly. If we are testing the API
surface on the servers, the clients should be able to correctly test all
of this via a mock of those API returns, which would let us separate
concerns here and keep the client tests close to their code as unit tests.
We're actually actively trying to figure out what can migrate out of
tempest back to the integrated projects, so that we get our biggest bang
for our buck.
Also, realize in a tempest environment there is only going to be the
latest version of the clients, so this is going to massively reduce your
test environment from what you have today.
-Sean
--
Sean Dague
http://dague.net
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
