Improving the QA test suite in multi-region would allow us to catch cases where a change has been made without considering the implications of multi-region support.
It would not however identify cases such as missing command line options. Given that the test suite identifies a set of operations with an expected result, if the scenario has been missed, it would not have a test case and therefore it would not be detected. Catching these is more of a code/spec review problem. I'd suggest as a first step to do a quick analysis with the infra test team - on what multi-region scenarios are tested in the gate currently - what are the gaps with the current CLIs / APIs - raise bugs where there are gaps - what additional test cases need to be implemented to catch regressions Tim > -----Original Message----- > From: Mathieu Gagné [mailto:mga...@internap.com] > Sent: 18 August 2015 23:48 > To: openstack-operators@lists.openstack.org > Subject: [Openstack-operators] Improving quality of multi-region support > > Hi, > > This email is a follow-up on the discussion we are currently having in the > LDT (Large Deployment Team) session at the OpenStack Ops Meetup. [1] > > We collectively identified a lack test coverage around multi-region support. > A region being the concept found in Keystone catalog, not the > geographically concept. > > Issues identified revolved around configs changes (without proper release > notes), lack of support in OpenStack clients [2] or maybe OpenStack clients > within services themselves. > > Some issues got fixed after bug reports were opened, this is great. > > However I feel like most were found out too late in the process when it > could have been detected much earlier in the QA process. > > I would like to start a discussion around multi-region test coverage and how > we can improve the overall quality around multi-region support. > > > [1] https://etherpad.openstack.org/p/PAO-ops-large-deployments > [2] https://bugs.launchpad.net/python-ceilometerclient/+bug/1439553 > > -- > Mathieu > > _______________________________________________ > OpenStack-operators mailing list > OpenStack-operators@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators _______________________________________________ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
