On 4/6/15 12:49 PM, David Stanek wrote:
Exactly. This is the direction I have been going. Functional tests are
written using the public APIs using the client.
I would also add that I don't like that the Keystone unit tests are so
database heavy. I would not want MySQL or ant RDBMS to be a
requirement for running the tests.
is that because you'd prefer that the unit tests were more isolated, or
just that an external service is being used?
I've done some work with extensive mocking of SQL databases;
specifically mocking at the ORM level. It is nice when you get it to
run, but it's also a much bigger job to write fine-grained mocks like
that, the mocks can be brittle in relation to the code they're
targeting, and you also need to come up with some solution to actually
functional test your database access code.
I tend to think that having a mysqld service running is the lesser of
two evils and you get a lot more code coverage by going all the way out
to the DB.
On Mon, Apr 6, 2015, 12:42 Morgan Fainberg <morgan.fainb...@gmail.com
<mailto:morgan.fainb...@gmail.com>> wrote:
> On Apr 6, 2015, at 09:20, Mike Bayer <mba...@redhat.com
<mailto:mba...@redhat.com>> wrote:
>
>
>
>> On 4/6/15 12:06 PM, Clint Byrum wrote:
>> Excerpts from Boris Bobrov's message of 2015-04-03 18:29:08 -0700:
>>>> On Saturday 04 April 2015 03:55:59 Morgan Fainberg wrote:
>>>> I am looking forward to the Liberty cycle and seeing the
special casing we
>>>> do for SQLite in our migrations (and elsewhere). My
inclination is that we
>>>> should (similar to the deprecation of eventlet) deprecate
support for
>>>> SQLite in Keystone. In Liberty we will have a full functional
test suite
>>>> that can (and will) be used to validate everything against
much more real
>>>> environments instead of in-process “eventlet-like”
test-keystone-services;
>>>> the “Restful test cases” will no longer be part of the
standard unit tests
>>>> (as they are functional testing). With this change I’m
inclined to say
>>>> SQLite (being the non-production usable DB) what it is we
should look at
>>>> dropping migration support for SQLite and the custom
work-arounds.
>>>>
>>>> Most deployers and developers (as far as I know) use devstack
and MySQL or
>>>> Postgres to really suss out DB interactions.
>>>>
>>>> I am looking for feedback from the community on the general
stance for
>>>> SQLite, and more specifically the benefit (if any) of
supporting it in
>>>> Keystone.
>>> +1. Drop it and clean up tons of code used for support of
sqlite only.
>>>
>>> Doing tests with mysql is as easy, as with sqlite ("mysqladmin
drop -f;
>>> mysqladmin create" for "reset"), and using it by default will
finally make
>>> people test their code on real rdbmses.
>> Please please please be careful with that and make sure the
database
>> name is _always_ random in tests... or even better, write a
fixture to
>> spin up a mysqld inside a private tempdir. That would be a
really cool
>> thing for oslo.db to provide actually.
>>
>> I'm just thinking some poor sap runs the test suite with the wrong
>> .my.cnf in the wrong place and <poof> there went keystone's db.
> The standard approach here is that tests based on the oslo.db
approach by default connect using a username openstack_citest on
localhost, which is then used to create databases of random names.
If you base your database tests on oslo.db, you get that right
now. This username/host/etc is also configurable through
environment variables. I don't see how my.cnf is involved in this
nor how it would impact someone's keystone database, unless we're
talking about tests that happen to load down and/or crash the
whole database server.
>
> spinning up a whole mysqld seems really heavy-handed and
unnecessary. Not to mention the tests run on other backends as
well such as Postgresql.
>
The reasons outlined by both Clint and Mike are why we won't be
attempting this until we are happy with our functional test suite.
But once we are happy dropping SQLite is on the table. The way I
see it the functional tests should be performed against a real
keystone server, even if it is one spun up for testing specifically.
Per test db creation / other such stuff will be part of that
discussion.
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
