Hi Timur, The current status of this work is, 1) The QA user story for destructive testing of OpenStack cloud [1] is merged . 2) The spec for a new framework which will focus on HA/failover and destructive testing [2] has no update since Nov 30 2016. 3) The commit for the new repository [3] has abandoned due to no update after Nov 29 2016.
Currently, I am working on 3rd party destructive/HA testing CI for Masakari[4] and very much interested in this work. I will keep working on [1] with PWG. Please let me know your plans for [2], and [3]. If it is difficult for you to continue, I would love to continue your work on [2], and [3]. [1] https://review.openstack.org/#/c/396142 [2] https://review.openstack.org/#/c/399618 [3] https://review.openstack.org/#/c/374667 [4] https://wiki.openstack.org/wiki/Masakari --- Regards, Sampath On Mon, Nov 28, 2016 at 6:37 AM, Timur Nurlygayanov <tnurlygaya...@mirantis.com> wrote: > Hi team, > > here is a short update: > > 1) The QA user story for destructive testing of OpenStack cloud is on review > [1]. > 2) The spec for a new framework which will focus on HA/failover and > destructive testing is no review [2]. > 3) The commit for the new repository is on review [3] as well. > > [1] https://review.openstack.org/#/c/396142 > [2] https://review.openstack.org/#/c/399618 > [3] https://review.openstack.org/#/c/374667 > > > On Fri, Oct 14, 2016 at 3:47 AM, Ghanshyam Mann > <ghanshyam.m...@nectechnologies.in> wrote: >> >> I like os-faults library which can provide the abstraction over different >> destructive actions like reboot/poweroff node etc. >> >> >> >> But not much clear about Stepler that what all tests it will contain. >> Tempest do have scenario tests which can hit the production env with >> significant way of testing scenario. >> >> It can cover the end user scenario also which involves the interaction of >> public OpenStack APIs and always in enhancement state by adding more and >> more scenario tests. >> >> >> >> Few query over Stepler as separate project: >> >> 1. Is Stepler will cover only tests which hits the node level >> action(reboot, HA etc)? >> >> 2. This should not mix the scenario tests which are in Tempest scope >> because that can make confusion for developers (where to add scenario tests) >> as well as for tester(from where to run what scenario tests). >> >> 3. How we make sure those tests run fine or at least run while >> adding. >> >> a. I think devstack enhancement for multi-node etc can do this as >> mentioned by you also. >> >> b. If so then why not adding those tests in Tempest only using >> os-faults lib ? >> >> >> >> Overall I feel os-faults as lib is really nice idea but tests scope can >> go in Tempest under HW_scenario (or something else) till it does not break >> basic principle of Tempest. >> >> >> >> Thanks >> >> gmann >> >> >> >> From: Timur Nurlygayanov [mailto:tnurlygaya...@mirantis.com] >> Sent: 06 October 2016 20:09 >> To: OpenStack Development Mailing List (not for usage questions) >> <openstack-dev@lists.openstack.org> >> Subject: Re: [openstack-dev] [QA] The end-user test suite for OpenStack >> clusters >> >> >> >> Ken, it is a good idea! >> >> >> >> The plan is to develop os-faults as a library which will be able >> >> to manage cluster nodes and OpenStack services on these nodes. >> >> It is a good idea to add some Tempest tests which will use >> >> os-faults library as well for some API tests. >> >> >> >> The Stepler framework [1] will use os-faults to perform all destructive >> >> actions in the clouds (the reboot of nodes, restart of OpenStack services, >> >> enable/disable network interfaces or some firewall rules and etc). >> >> >> >> We need to get +1 from you in [1] to create the repository with >> >> advanced end-user scenario tests. >> >> >> >> Thank you! >> >> >> >> [1] https://review.openstack.org/#/c/374667/ >> >> >> >> On Tue, Oct 4, 2016 at 8:53 PM, Yaroslav Lobankov <yloban...@mirantis.com> >> wrote: >> >> Hi Ken, >> >> >> >> OS-Faults doesn't have any scenarios in the tree yet (the project is two >> months old), but you can find some examples of the use in the >> os-faults/examples directory. >> >> >> >> Regards, >> >> Yaroslav Lobankov. >> >> >> >> On Tue, Oct 4, 2016 at 8:02 PM, Ken'ichi Ohmichi <ken1ohmi...@gmail.com> >> wrote: >> >> Hi Timur, >> >> Thanks for your explanation. >> >> >> 2016-09-29 6:22 GMT-07:00 Timur Nurlygayanov <tnurlygaya...@mirantis.com>: >> > >> >> I am guessing the above "restart nodes" is for verifying each >> >> OpenStack service restarts successfully, right? >> > >> > Yes, this is right. And we also will check that HA logic for these >> > services works correctly (for example, rescheduling of L3 Neutron >> > agents for networks). >> > >> >> But these service scripts are provided by distributors, and Devstack >> >> itself doesn't contain service scripts IIUC. >> >> So I'd like to know how to verify it on Devstack clouds. >> > >> > Yes, DevStack doesn't support many scenarios which are actual >> > and should be supported on the production clouds. >> > It will be not possible to run all advanced test scenarios for DevStack >> > clouds, >> > just because DevStack can't deploy OpenStack cloud with 3 controllers >> > now (so, probably it will be possible in the future). >> > >> > Of course, some advanced scenarios will support DevStack clouds, >> > for example, some test scenarios which are based on customer-found >> > issues from the real production clouds, like upload of the large images >> > (100+ Gb) >> > to Glance with Swift backend. Such cases are important for verification >> > of >> > pre-production environments, but not very important for CI gate jobs. >> > >> > It is also important to note that in these advanced cases we are >> > targeting >> > to check not only the logic of Python code, but also the correct >> > configuration >> > of all OpenStack components on some pre-production OpenStack clusters. >> >> I guessed some part of os-faults can be moved to Tempest if os-faults >> contains API tests for enabling/disabling OpenStack services. >> Then, os-faults would be able to concentrate on more destructive tests >> like rebooting physical nodes, etc. >> However, I could not find any actual scenarios on current os-faults >> (https://github.com/openstack/os-faults). >> That seems to just contain some abstraction layers and unit tests. Can >> we see actual test scenarios of os-faults ? >> Maybe I missed something. >> >> Thanks >> Ken Ohmichi >> >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> >> >> -- >> >> >> >> Timur, >> >> Senior QA Manager >> >> OpenStack Projects >> >> Mirantis Inc >> >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > > -- > > Timur, > QA Manager > OpenStack Projects > Mirantis Inc > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
