I agree with Mike's concerns, and propose to make these limitations (4 weeks before FF for OS upgrades, 2 weeks for upgrades of key dependencies -- RabbitMQ, MCollective, Puppet, MySQL, PostgreSQL, anything else?) official for 10.0/Newton.
For 9.0/Mitaka, it is too late to impose them, so we just have to be very careful and conservative with this upgrade. First of all, we need to have a green BVT before and after switching to the CentOS 7.2 repo snapshot, so while I approved the spec, we can't move forward with this until BVT is green again, and right now it's red: https://ci.fuel-infra.org/job/9.0.fuel_community.ubuntu.bvt_2/ If we get it back to green but it becomes red after the upgrade, you must switch back to CentOS 7.1 *immediately*. If you are able to stick to this plan, there is still time to complete the transition today without requiring an FFE. -- Dmitry Borodaenko On Wed, Mar 02, 2016 at 05:53:53PM +0000, Mike Scherbakov wrote: > Formally, we can merge it today. Historically, every update of OS caused us > instability for some time: from days to a couple of month. > Taking this into account and number of other exceptions requested, overall > stability of code, my opinion would be to postpone this to 10.0. > > Also, I'd suggest to change the process, and have freeze date for all OS > updates no later than a month before official FF date. This will give us > time to stabilize, and ensure that base on which all new code is being > developed is stable when approaching FF. > > I'd also propose to have freeze for major upgrades of 3rd party packages no > later than 2 weeks before FF, which Fuel depends heavily upon. For > instance, such will include RabbitMQ, MCollective, Puppet. > > On Wed, Mar 2, 2016 at 7:34 AM Igor Marnat <imar...@mirantis.com> wrote: > > > Igor, > > couple of points from my side. > > > > CentOS 7.2 will be getting updates for several more months, and we have > > snapshots and all the mechanics in place to switch to the next version when > > needed. > > > > Speaking of getting this update into 9.0, we actually don't need FFE, we > > can merge remaining staff today. It has enough reviews, so if you add your > > +1 today, we don't need FFE. > > > > https://review.openstack.org/#/c/280338/ > > https://review.fuel-infra.org/#/c/17400/ > > > > > > > > Regards, > > Igor Marnat > > > > On Wed, Mar 2, 2016 at 6:23 PM, Dmitry Teselkin <dtesel...@mirantis.com> > > wrote: > > > >> Igor, > >> > >> Your statement about updates for 7.2 isn't correct - it will receive > >> updates, because it's the latest release ATM. There is *no* pinning inside > >> ISO, and the only place where it was 8.0 were docker containers just > >> because we had to workaround some issues. But there are no docker > >> containers in 9.0, so that's not the case. > >> The proposed solution to switch to CentOS-7.2 in fact is based on > >> selecting the right snapshot with packages. There is no pinning in ISO (it > >> was in earlier versions of the spec but was removed). > >> > >> On Wed, Mar 2, 2016 at 6:11 PM, Igor Kalnitsky <ikalnit...@mirantis.com> > >> wrote: > >> > >>> Dmitry, Igor, > >>> > >>> > Very important thing is that CentOS 7.1 which master node is based now > >>> > don't get updates any longer. > >>> > >>> If you are using "fixed" release you must be ready that you won't get > >>> any updates. So with CentOS 7.2 the problem still the same. > >>> > >>> However, let's wait for Fuel PTL decision. I only shared my POV: > >>> that's not a critical feature, and taking into account the risks of > >>> regression - I'd prefer to do not accept it in 9.0. > >>> > >>> Regards, > >>> Igor > >>> > >>> On Wed, Mar 2, 2016 at 4:42 PM, Igor Marnat <imar...@mirantis.com> > >>> wrote: > >>> > Igor, > >>> > please note that this is pretty much not like update of master node > >>> which we > >>> > had in 8.0. This is minor _update_ of CentOS from 7.1 to 7.2 which team > >>> > tested for more than 2 months already. > >>> > > >>> > We don't expect it to require any additional efforts from core or qa > >>> team. > >>> > > >>> > Very important thing is that CentOS 7.1 which master node is based now > >>> don't > >>> > get updates any longer. Updates are only provided for CentOS 7.2. > >>> > > >>> > So we'll have to switch CentOS 7.1 to CentOS 7.2 anyways. > >>> > > >>> > We can do it now for more or less free, later in release cycle for > >>> higher > >>> > risk and QA efforts and after the release for 2x price because of > >>> additional > >>> > QA cycle we'll need to pass through. > >>> > > >>> > > >>> > > >>> > Regards, > >>> > Igor Marnat > >>> > > >>> > On Wed, Mar 2, 2016 at 2:57 PM, Dmitry Teselkin < > >>> dtesel...@mirantis.com> > >>> > wrote: > >>> >> > >>> >> Hi Igor, > >>> >> > >>> >> Postponing this till Fuel 10 means we have to elaborate a plan to do > >>> such > >>> >> upgrade for Fuel 9 after the release - the underlying system will not > >>> get > >>> >> updated on it's own, and the security issues will not close > >>> themselves. The > >>> >> problem here is that such upgrade of deployed master node requires a > >>> lot of > >>> >> QA work also. > >>> >> > >>> >> Since we are not going to update package we build on our own (they > >>> still > >>> >> targeted 7.1) switching master node base to CentOS-72 is not that > >>> dangerous > >>> >> as it seems. > >>> >> > >>> >> On Wed, Mar 2, 2016 at 1:54 PM, Igor Kalnitsky < > >>> ikalnit...@mirantis.com> > >>> >> wrote: > >>> >>> > >>> >>> Hey Dmitry, > >>> >>> > >>> >>> No offence, but I rather against that exception. We have too many > >>> >>> things to do in Mitaka, and moving to CentOS 7.2 means > >>> >>> > >>> >>> * extra effort from core team > >>> >>> * extra effort from qa team > >>> >>> > >>> >>> Moreover, it might block development by introducing unpredictable > >>> >>> regressions. Remember 8.0? So I think it'd be better to reduce risk > >>> of > >>> >>> regressions that affects so many developers by postponing CentOS 7.2 > >>> >>> till Fuel 10. > >>> >>> > >>> >>> Thanks, > >>> >>> Igor > >>> >>> > >>> >>> > >>> >>> On Mon, Feb 29, 2016 at 7:13 PM, Dmitry Teselkin < > >>> dtesel...@mirantis.com> > >>> >>> wrote: > >>> >>> > I'd like to ask for a feature freeze exception for switching to > >>> >>> > CentOS-7.2 > >>> >>> > feature [0]. > >>> >>> > > >>> >>> > CentOS-7.2 ISO's have been tested periodically since the beginning > >>> of > >>> >>> > the > >>> >>> > year, and all major issues were addressed / fixed at the moment. > >>> During > >>> >>> > the > >>> >>> > last weekend I've made 70 BVT runs to verify that the solution > >>> [2] for > >>> >>> > the > >>> >>> > last issue - e1000 transmit unit hangs works. And it works, 0 > >>> tests of > >>> >>> > 35 > >>> >>> > failed [3]. > >>> >>> > > >>> >>> > Benefits of switching to CentOS-7.2 are quite obvious - we will > >>> return > >>> >>> > to > >>> >>> > latest supported CentOS release, will fix a lot of bugs / security > >>> >>> > issues > >>> >>> > [4] and will make further updates easier. > >>> >>> > > >>> >>> > Risk of regression still exists, but it's quite low, 35 successful > >>> BVTs > >>> >>> > can't be wrong. > >>> >>> > > >>> >>> > To finish that feature the following should be done: > >>> >>> > * review and merge e1000 workaround [2] > >>> >>> > * review and merge spec [0] > >>> >>> > * review and merge request that switches build CI to CentOS-7.2 [5] > >>> >>> > > >>> >>> > I expect the last day it could be done is March, 4. > >>> >>> > > >>> >>> > [0] https://review.openstack.org/#/c/280338/ > >>> >>> > [1] https://bugs.launchpad.net/fuel/+bug/1526544 > >>> >>> > [2] https://review.openstack.org/#/c/285306/ > >>> >>> > [3] > >>> https://etherpad.openstack.org/p/r.1c4cfee8185326d6922d6c9321404350 > >>> >>> > [4] > >>> https://etherpad.openstack.org/p/r.a7fe0b575d891ed81206765fa5be6630 > >>> >>> > [5] https://review.fuel-infra.org/#/c/17400/ > >>> >>> > > >>> >>> > > >>> >>> > -- > >>> >>> > Thanks, > >>> >>> > Dmitry Teselkin > >>> >>> > Mirantis > >>> >>> > http://www.mirantis.com > >>> >>> > > >>> >>> > > >>> >>> > > >>> __________________________________________________________________________ > >>> >>> > OpenStack Development Mailing List (not for usage questions) > >>> >>> > Unsubscribe: > >>> >>> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >>> >>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >>> >>> > > >>> >>> > >>> >>> > >>> >>> > >>> __________________________________________________________________________ > >>> >>> OpenStack Development Mailing List (not for usage questions) > >>> >>> Unsubscribe: > >>> >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >>> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >>> >> > >>> >> > >>> >> > >>> >> > >>> >> -- > >>> >> Thanks, > >>> >> Dmitry Teselkin > >>> >> Mirantis > >>> >> http://www.mirantis.com > >>> >> > >>> >> > >>> __________________________________________________________________________ > >>> >> OpenStack Development Mailing List (not for usage questions) > >>> >> Unsubscribe: > >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >>> >> > >>> > > >>> > > >>> > > >>> __________________________________________________________________________ > >>> > OpenStack Development Mailing List (not for usage questions) > >>> > Unsubscribe: > >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >>> > > >>> > >>> > >>> __________________________________________________________________________ > >>> OpenStack Development Mailing List (not for usage questions) > >>> Unsubscribe: > >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >>> > >> > >> > >> > >> -- > >> Thanks, > >> Dmitry Teselkin > >> Mirantis > >> http://www.mirantis.com > >> > >> __________________________________________________________________________ > >> OpenStack Development Mailing List (not for usage questions) > >> Unsubscribe: > >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > >> > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > -- > Mike Scherbakov > #mihgen > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
