On Thu, Nov 24, 2016 at 11:08 AM, Juan Antonio Osorio <jaosor...@gmail.com> wrote: > I don't have a strong opinion about any option, as long as we have something > in place I'm happy. > > But regarding option 1.A: what would be done for newton once these templates > are moved to t-h-t. Would they be backported? What about mitaka?
Newton: yes: we'll have to backport https://review.openstack.org/#/c/402119/ and that's it! Mitaka: we never supported and ran scenarios, so no problem here. I started a first iteration: https://review.openstack.org/#/q/topic:tripleo-ci/scenarios Feel free to review and give any feedback. Thanks, > > On 24 Nov 2016 17:55, "Carlos Camacho Gonzalez" <ccama...@redhat.com> wrote: >> >> I think would be cool to go with option, +1 to 1.A >> >> IMHO, >> - Easier to read. >> - Easier to maintain. >> - We don't make backports, instead we guarantee backwards compatibility. >> - We'll re-use experience from Puppet OpenStack CI. >> >> On Wed, Nov 23, 2016 at 10:13 PM, Giulio Fidente <gfide...@redhat.com> >> wrote: >> > hi Emilien, >> > >> > thanks for putting some thought into this. We have a similar problem to >> > test >> > RGW which was only added in Newton. >> > >> > >> > On 11/23/2016 03:02 AM, Emilien Macchi wrote: >> >> >> >> == Context >> >> >> >> In Newton we added new multinode jobs called "scenarios". >> >> The challenged we tried to solve was "how to test the maximum of >> >> services without overloading the nodes that run tests". >> >> >> >> Each scenarios deploys a set of services, which allows us to >> >> horizontally scale the number of scenarios to increase the service >> >> testing coverage. >> >> See the result here: >> >> https://github.com/openstack-infra/tripleo-ci#service-testing-matrix >> >> >> >> To implement this model, we took example of Puppet OpenStack CI: >> >> https://github.com/openstack/puppet-openstack-integration#description >> >> We even tried to keep consistent the services/scenarios relations, so >> >> it's consistent and easier to maintain. >> >> >> >> Everything was fine until we had to add new services during Ocata >> >> cycles. >> >> Because tripleo-ci repository is not branched, adding Barbican service >> >> in the TripleO environment for scenario002 would break Newton CI jobs. >> >> During my vacations, the team created a new scenario, scenario004, >> >> that deploys Barbican and that is only run for Ocata jobs. >> >> I don't think we should proceed this way, and let me explain why. >> >> >> >> == Problem >> >> >> >> How to scale the number of services that we test without increasing >> >> the number of scenarios and therefore the complexity of maintaining >> >> them on long-term. >> >> >> >> >> >> == Solutions >> >> >> >> The list is not exhaustive, feel free to add more. >> >> >> >> 1) Re-use experience from Puppet OpenStack CI and have environments >> >> that are in a branched repository. >> >> environments. >> >> In Puppet OpenStack CI, the repository that deploys environments >> >> (puppet-openstack-integration) is branched. So if puppet-barbican is >> >> ready to be tested in Ocata, we'll patch >> >> puppet-openstack-integration/master to start testing it and it won't >> >> break stable jobs. >> >> Like this, we were successfully able to maintain a fair number of >> >> scenarios and keep our coverage increasing over each cycle. >> >> >> >> I see 2 sub-options here: >> >> >> >> a) Move CI environments and pingtest into >> >> tripleo-heat-templates/environments/ci/(scenarios|pingtest). This repo >> >> is branched and we could add a README to explain these files are used >> >> in CI and we don't guarantee they would work outside TripleO CI tools. >> >> b) Branch tripleo-ci repository. Personally I don't like this solution >> >> because a lot of patches in this repo are not related to OpenStack >> >> versions, which means we would need to backport most of the things >> >> from master. >> > >> > >> > I'd +1 this idea. It sounds like we could make the scenarios generic >> > enough >> > to be usable also outside CI? Maybe they can serve as samples? >> > -- >> > Giulio Fidente >> > GPG KEY: 08D733BA >> > >> > >> > >> > __________________________________________________________________________ >> > OpenStack Development Mailing List (not for usage questions) >> > Unsubscribe: >> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Emilien Macchi __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
