Igor,
I wonder what are the benefits of using Rally then? We can run whatever we > want by means of MCollective or Ansible. Is there something that could help > us? I don't know, maybe some dashboard integration with per test results or > running by tests by tag? Benefits are in the Rally framework, engine and integrated tooling, that are doing very hard things to provide simple interfaces for writing simple plugins that are emulating complicated test cases. The major benefits are next: *1) Generalization* 1.1) One tool with one reporting system and one output format for all kinds of testing strategies (functional, load, perf, scale, ...) 1.2) One set of plugins (code) that can be used to generate all kinds of testing strategies 1.3.) One API for all kinds of testing strategies *2) Simplicity * 2.1) Plugins are really simple to write, usually requires one method to be implemented 2.2) Auto discovery: adding plugins == add code in special (or specified) directory *3) Reusability & Data Driven approach: * 3.1) Split code (plugins) & tests cases (yaml files) 3.2) Test cases is mixture of different plugins 3.3) Plugins accept arguments *4) Integrated tooling* 4.1) All results are persisted in Rally DB and you can access it in any moment 4.2) Results can be exported in different formats (you can write even own plugins for simplifying integration) 4.3) Detailed HMTL reports with task results overview and trends Before switching Fuel from ostf to rally, I would like to see feature > parity comparison. It's very necessary to understand how much work we need > to spend to rewrite all our tests in rally way. Totally agree, let's do it. Best regards, Boris Pavlovic On Mon, Jun 27, 2016 at 8:58 AM, Vladimir Kuklin <vkuk...@mirantis.com> wrote: > +1 to initial suggestion, but I guess we need to have a full feature > equality (e.g. HA tests for mysql and rabbitmq replication) before > switching to Rally. > > On Mon, Jun 27, 2016 at 6:17 PM, Sergii Golovatiuk < > sgolovat...@mirantis.com> wrote: > >> Hi, >> >> Before switching Fuel from ostf to rally, I would like to see feature >> parity comparison. It's very necessary to understand how much work we need >> to spend to rewrite all our tests in rally way. >> >> >> >> -- >> Best regards, >> Sergii Golovatiuk, >> Skype #golserge >> IRC #holser >> >> On Mon, Jun 27, 2016 at 4:32 PM, Alexander Kostrikov < >> akostri...@mirantis.com> wrote: >> >>> Hello, everybody! >>> Hello, Alex! >>> >I thought Rally was more for benchmarking. Wouldn't Tempest make more >>> sense? >>> Rally is a good tool with nice api/usage/extensibility. >>> I really liked "up and running tests in 5 minutes" in Rally with clear >>> picture of what I am doing. >>> So, I 100% for a Rally as a QA. >>> >>> Another note: >>> We will need to implement some HA tests, probably not in Rally. >>> >>> On Mon, Jun 27, 2016 at 4:57 PM, Andrey Kurilin <akuri...@mirantis.com> >>> wrote: >>> >>>> >>>> >>>> On Mon, Jun 27, 2016 at 4:46 PM, Igor Kalnitsky < >>>> ikalnit...@mirantis.com> wrote: >>>> >>>>> >>>>> > On Jun 27, 2016, at 16:23, Alex Schultz <aschu...@mirantis.com> >>>>> wrote: >>>>> > >>>>> > I thought Rally was more for benchmarking. Wouldn't Tempest make >>>>> more sense? >>>>> >>>>> According to Rally wiki page [1], it seems they have a verification >>>>> layer (Tempest so far). Hm, I wonder does it mean we will need to rewrite >>>>> our scenarios for Tempest? >>>>> >>>>> >>>> Rally consists of two main components: Rally Task and Rally >>>> Verification. They are totally separated. >>>> Task component is fully pluggable and you can run there whatever you >>>> want in whatever you want way. >>>> Verification component is hardcoded now. It was designed for >>>> managing(install, configure) and launching(execute and store results) >>>> Tempest. But we have a spec to make this component pluggable too. >>>> >>>> >>>>> - igor >>>>> >>>>> >>>>> [1] https://wiki.openstack.org/wiki/Rally >>>>> >>>>> __________________________________________________________________________ >>>>> OpenStack Development Mailing List (not for usage questions) >>>>> Unsubscribe: >>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >>>> >>>> >>>> >>>> -- >>>> Best regards, >>>> Andrey Kurilin. >>>> >>>> >>>> __________________________________________________________________________ >>>> OpenStack Development Mailing List (not for usage questions) >>>> Unsubscribe: >>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> >>> >>> >>> -- >>> >>> Kind Regards, >>> >>> Alexandr Kostrikov, >>> >>> Mirantis, Inc. >>> >>> 35b/3, Vorontsovskaya St., 109147, Moscow, Russia >>> >>> >>> Tel.: +7 (495) 640-49-04 >>> Tel.: +7 (925) 716-64-52 <%2B7%20%28906%29%20740-64-79> >>> >>> Skype: akostrikov_mirantis >>> >>> E-mail: akostri...@mirantis.com <elogut...@mirantis.com> >>> >>> *www.mirantis.com <http://www.mirantis.ru/>* >>> *www.mirantis.ru <http://www.mirantis.ru/>* >>> >>> >>> __________________________________________________________________________ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > > -- > Yours Faithfully, > Vladimir Kuklin, > Fuel Library Tech Lead, > Mirantis, Inc. > +7 (495) 640-49-04 > +7 (926) 702-39-68 > Skype kuklinvv > 35bk3, Vorontsovskaya Str. > Moscow, Russia, > www.mirantis.com <http://www.mirantis.ru/> > www.mirantis.ru > vkuk...@mirantis.com > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
