Thanks, Steve. I suggested a new directory because we really need to
have more complete tests of the client libs since they are not tied to
particular OpenStack releases and we claim the current libs should work
with older releases. That said, I did not realize the intent was to do
more than test the client libraries. If the intent of the follow-ons to
this patch is to be more scenario-like rather than just increasing
coverage for the keystoneclient api then it makes sense to go in
scenario/keystone.
I still think we should have more client lib coverage than the haphazard
usage we will get from scenario but a new directory could wait until
such code starts to exist.
-David
On 10/18/2013 11:34 AM, Steven Hardy wrote:
Hi all,
Starting a thread to discuss $subject, as requested in:
https://review.openstack.org/#/c/51558/
First a bit of background. I wrote a keystoneclient patch, and ayoung
stated he'd like it tested via tempest before he'd ack it:
https://review.openstack.org/#/c/48462/
So I spoke to ayoung and dkranz on IRC, showing them my local tests for the
patch. dkranz suggested creating a "client_lib" directory, where we could
build out a more comprehensive set of tests over time, adding to the inital
tests related to keystone trusts client additions.
A couple of things to note:
- These are end-to-end tests, designed to test not only the client, but
also the API and keystone backend functionality. So arguably this could
just be a scenario test, e.g scenario/keystone/test_v3_auth.py
- The intention is to excercise logic which is hard to fully test with
unit or integration tests, and to catch issues like incompatibility
between client and API - e.g keystoneclient tests may pass, but we need
to make sure the client actually works against the real keystone API.
Working on Heat has given me a pretty good insight into the python-*client
API's, as we use them to orchestrate actions with every openstack service;
IMO anything we can do to make these interfaces more robust (and catch
bugs, several of which I found already while writing these tests) is a
good-thing (tm).
I'd welcome feedback on the patch above, and what will be the most
acceptable approach to the tempest team for adding these tests.
More links:
https://review.openstack.org/#/c/51559/
https://review.openstack.org/#/c/51560/
https://blueprints.launchpad.net/tempest/+spec/keystoneclient-api
Thanks!
Steve
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
