On 02/03, Ivan Kolodyazhny wrote: > I'll try to implement such scenario and step-by-step guideline soon. >
That would be fantastic!! Thank you very much Looking forward to it. :-) Cheers, Gorka. > Regards, > Ivan Kolodyazhny, > http://blog.e0ne.info/ > > On Wed, Mar 2, 2016 at 5:16 PM, Eric Harney <ehar...@redhat.com> wrote: > > > On 03/02/2016 10:07 AM, Ivan Kolodyazhny wrote: > > > Eric, > > > > > > For now, we test Cinder API with some concurrency only with Rally, so, > > IMO, > > > it's reasonable get more scenarios for API races fixes. > > > > > > It's not a hard task to implement new scenarios, they are pretty simple: > > > [11] and [12] > > > > > > > Sure, these are simple, but I think it's nowhere near that simple to > > write a scenario which will prove that "remove API races" works correctly. > > > > > [11] > > > > > https://github.com/openstack/rally/blob/master/rally/plugins/openstack/scenarios/cinder/volumes.py#L535 > > > [12] > > > > > https://github.com/openstack/rally/blob/master/rally-jobs/cinder.yaml#L516 > > > > > > Regards, > > > Ivan Kolodyazhny, > > > http://blog.e0ne.info/ > > > > > > On Wed, Mar 2, 2016 at 4:50 PM, Eric Harney <ehar...@redhat.com> wrote: > > > > > >> On 03/02/2016 09:36 AM, Ivan Kolodyazhny wrote: > > >>> Eric, > > >>> > > >>> There are Gorka's patches [10] to remove API Races > > >>> > > >>> > > >>> [10] > > >>> > > >> > > https://review.openstack.org/#/q/project:openstack/cinder+branch:master+topic:fix/api-races-simplified > > >>> > > >>> Regards, > > >>> Ivan Kolodyazhny, > > >>> http://blog.e0ne.info/ > > >>> > > >> > > >> So the second part of my question is, is writing a Rally job to prove > > >> out that code a reasonable task? > > >> > > >> How hard is that to do and what does it look like? > > >> > > >>> On Wed, Mar 2, 2016 at 4:27 PM, Eric Harney <ehar...@redhat.com> > > wrote: > > >>> > > >>>> On 03/02/2016 06:25 AM, Ivan Kolodyazhny wrote: > > >>>>> Hi Team, > > >>>>> > > >>>>> Here are my thoughts and proposals how to make Cinder testing process > > >>>>> better. I won't cover "3rd party CI's" topic here. I will share my > > >>>> opinion > > >>>>> about current and feature jobs. > > >>>>> > > >>>>> > > >>>>> Unit-tests > > >>>>> > > >>>>> - Long-running tests. I hope, everybody will agree that unit-tests > > >>>> must > > >>>>> be quite simple and very fast. Unit tests which takes more than > > 3-5 > > >>>> seconds > > >>>>> should be refactored and/or moved to 'integration' tests. > > >>>>> Thanks to Tom Barron for several fixes like [1]. IMO, we it would > > be > > >>>>> good to have some hacking checks to prevent such issues in a > > future. > > >>>>> > > >>>>> - Tests coverage. We don't check it in an automatic way on gates. > > >>>>> Usually, we require to add some unit-tests during code review > > >>>> process. Why > > >>>>> can't we add coverage job to our CI and do not merge new patches, > > >> with > > >>>>> will decrease tests coverage rate? Maybe, such job could be voting > > >> in > > >>>> a > > >>>>> future to not ignore it. For now, there is not simple way to check > > >>>> coverage > > >>>>> because 'tox -e cover' output is not useful [2]. > > >>>>> > > >>>>> > > >>>>> Functional tests for Cinder > > >>>>> > > >>>>> We introduced some functional tests last month [3]. Here is a patch > > to > > >>>>> infra to add new job [4]. Because these tests were moved from > > >>>> unit-tests, I > > >>>>> think we're OK to make this job voting. Such tests should not be a > > >>>>> replacement for Tempest. They even could tests Cinder with Fake > > Driver > > >> to > > >>>>> make it faster and not related on storage backends issues. > > >>>>> > > >>>>> > > >>>>> Tempest in-tree tests > > >>>>> > > >>>>> Sean started work on it [5] and I think it's a good idea to get them > > in > > >>>>> Cinder repo to run them on Tempest jobs and 3-rd party CIs against a > > >> real > > >>>>> backend. > > >>>>> > > >>>>> > > >>>>> Functional tests for python-brick-cinderclient-ext > > >>>>> > > >>>>> There are patches that introduces functional tests [6] and new job > > [7]. > > >>>>> > > >>>>> > > >>>>> Functional tests for python-cinderclient > > >>>>> > > >>>>> We've got a very limited set of such tests and non-voting job. IMO, > > we > > >>>> can > > >>>>> run them even with Cinder Fake Driver to make them not depended on a > > >>>>> storage backend and make it faster. I believe, we can make this job > > >>>> voting > > >>>>> soon. Also, we need more contributors to this kind of tests. > > >>>>> > > >>>>> > > >>>>> Integrated tests for python-cinderclient > > >>>>> > > >>>>> We need such tests to make sure that we won't break Nova, Heat or > > other > > >>>>> python-cinderclient consumers with a next merged patch. There is a > > >> thread > > >>>>> in openstack-dev ML about such tests [8] and proposal [9] to > > introduce > > >>>> them > > >>>>> to python-cinderclient. > > >>>>> > > >>>>> > > >>>>> Rally tests > > >>>>> > > >>>>> IMO, it would be good to have new Rally scenarios for every patches > > >> like > > >>>>> 'improves performance', 'fixes concurrency issues', etc. Even if we > > as > > >> a > > >>>>> Cinder community don't have enough time to implement them, we have to > > >> ask > > >>>>> for them in reviews, openstack-dev ML, file Rally bugs and blueprints > > >> if > > >>>>> needed. > > >>>>> > > >>>> > > >>>> Are there any recent examples of a fix like this recently where it > > would > > >>>> seem like a reasonable task to write a Rally scenario along with the > > >> patch? > > >>>> > > >>>> Not being very familiar with Rally (as I think most of us aren't), I'm > > >>>> having a hard time picturing this. > > >>>> > > >>>>> > > >>>>> [1] https://review.openstack.org/#/c/282861/ > > >>>>> [2] http://paste.openstack.org/show/488925/ > > >>>>> [3] https://review.openstack.org/#/c/267801/ > > >>>>> [4] https://review.openstack.org/#/c/287115/ > > >>>>> [5] https://review.openstack.org/#/c/274471/ > > >>>>> [6] https://review.openstack.org/#/c/265811/ > > >>>>> [7] https://review.openstack.org/#/c/265925/ > > >>>>> [8] > > >>>>> > > >>>> > > >> > > http://lists.openstack.org/pipermail/openstack-dev/2016-March/088027.html > > >>>>> [9] https://review.openstack.org/#/c/279432/ > > >>>>> > > >>>>> > > >>>>> Regards, > > >>>>> Ivan Kolodyazhny, > > >>>>> http://blog.e0ne.info/ > > >>>>> > > >> > > >> > > >> > > __________________________________________________________________________ > > >> OpenStack Development Mailing List (not for usage questions) > > >> Unsubscribe: > > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > >> > > > > > > > > > > > > > > __________________________________________________________________________ > > > OpenStack Development Mailing List (not for usage questions) > > > Unsubscribe: > > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
