Guys, I added patch to infra: https://review.openstack.org/#/c/143879/
That allows to run Rally against Mistral in gates. Best regards, Boris Pavlovic On Mon, Dec 22, 2014 at 4:25 PM, Anastasia Kuznetsova < [email protected]> wrote: > Dmitry, > > Now I see that my comments are not so informative, I will try to describe > environment and scenarios in more details. > > 1) *1 api 1 engine 1 executor *it means that there were 3 Mistral > processes running on the same box > 2) list-workbooks scenario was run when there were no workflow executions > at the same time, I will notice this your comment and I will measure time > in such situation, but I guess that it will take more time, the question is > as far as. > 3) 60 % of success means that only 60 % of number of times execution of > scenario 'list-workbooks' were successful, at the moment I have observed > only one type of error: > error connection to Rabbit : Error ConnectionError: ('Connection > aborted.', error(104, 'Connection reset by peer')) > 4) we don't know the durability criteria of Mistral and under what load > Mistral will 'die', we want to define the threshold. > > P.S. Dmitry, if you have any ideas/scenarios which you want to test, > please share them. > > On Sat, Dec 20, 2014 at 9:35 AM, Dmitri Zimine <[email protected]> > wrote: > >> Anastasia, any start is a good start. >> >> *> 1 api 1 engine 1 executor, list-workbooks* >> >> what exactly doest it mean: 1) is mistral deployed on 3 boxes with >> component per box, or all three are processes on the same box? 2) is >> list-workbooks test running while workflow executions going on? How many? >> what’s the character of the load 3) when it says 60% success what exactly >> does it mean, what kind of failures? 4) what is the durability criteria, >> how long do we expect Mistral to withstand the load. >> >> Let’s discuss this in details on the next IRC meeting? >> >> Thanks again for getting this started. >> >> DZ. >> >> >> On Dec 19, 2014, at 7:44 AM, Anastasia Kuznetsova < >> [email protected]> wrote: >> >> Boris, >> >> Thanks for feedback! >> >> > But I belive that you should put bigger load here: >> https://etherpad.openstack.org/p/mistral-rally-testing-results >> >> As I said it is only beginning and I will increase the load and change >> its type. >> >> >As well concurrency should be at least 2-3 times bigger than times >> otherwise it won't generate proper load and you won't collect >enough data >> for statistical analyze. >> > >> >As well use "rps" runner that generates more real life load. >> >Plus it will be nice to share as well output of "rally task report" >> command. >> >> Thanks for the advice, I will consider it in further testing and >> reporting. >> >> Answering to your question about using Rally for integration testing, as >> I mentioned in our load testing plan published on wiki page, one of our >> final goals is to have a Rally gate in one of Mistral repositories, so we >> are interested in it and I already prepare first commits to Rally. >> >> Thanks, >> Anastasia Kuznetsova >> >> On Fri, Dec 19, 2014 at 4:51 PM, Boris Pavlovic <[email protected]> >> wrote: >>> >>> Anastasia, >>> >>> Nice work on this. But I belive that you should put bigger load here: >>> https://etherpad.openstack.org/p/mistral-rally-testing-results >>> >>> As well concurrency should be at least 2-3 times bigger than times >>> otherwise it won't generate proper load and you won't collect enough data >>> for statistical analyze. >>> >>> As well use "rps" runner that generates more real life load. >>> Plus it will be nice to share as well output of "rally task report" >>> command. >>> >>> >>> By the way what do you think about using Rally scenarios (that you >>> already wrote) for integration testing as well? >>> >>> >>> Best regards, >>> Boris Pavlovic >>> >>> On Fri, Dec 19, 2014 at 2:39 PM, Anastasia Kuznetsova < >>> [email protected]> wrote: >>>> >>>> Hello everyone, >>>> >>>> I want to announce that Mistral team has started work on load and >>>> performance testing in this release cycle. >>>> >>>> Brief information about scope of our work can be found here: >>>> >>>> https://wiki.openstack.org/wiki/Mistral/Testing#Load_and_Performance_Testing >>>> >>>> First results are published here: >>>> https://etherpad.openstack.org/p/mistral-rally-testing-results >>>> >>>> Thanks, >>>> Anastasia Kuznetsova >>>> @ Mirantis Inc. >>>> >>>> _______________________________________________ >>>> OpenStack-dev mailing list >>>> [email protected] >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> >>> _______________________________________________ >>> OpenStack-dev mailing list >>> [email protected] >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> _______________________________________________ >> OpenStack-dev mailing list >> [email protected] >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> _______________________________________________ >> OpenStack-dev mailing list >> [email protected] >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
