On Thursday, January 29, 2015 at 11:40 AM, Fischer, Matt wrote: > > From: Morgan Fainberg <morgan.fainb...@gmail.com > (mailto:morgan.fainb...@gmail.com)> > Date: Thursday, January 29, 2015 at 12:26 PM > To: openstack-operators <openstack-operators@lists.openstack.org > (mailto:openstack-operators@lists.openstack.org)> > Subject: [Openstack-operators] [all] SQL Schema Downgrades: A Good Idea? > > >From an operator perspective I wanted to get input on the SQL Schema > >Downgrades. > > > >Today most projects (all?) provide a way to downgrade the SQL Schemas after > >you’ve upgraded. Example would be moving from Juno to Kilo and then back to > >Juno. There are some odd concepts >when handling a SQL migration downgrade > >specifically around the state of the data. A downgrade, in many cases, > >causes permanent and irrevocable data loss. When phrased like that (and > >>dusting off my deployer/operator hat) I would be hesitant to run a > >downgrade in any production, stagings, or even QA environment. > > > >In light of what a downgrade actually means I would like to get the views of > >the operators on SQL Migration Downgrades: > > > >1) Would you actually perform a programatic downgrade via the cli tools or > >would you just do a restore-to-last-known-good-before-upgrade (e.g. from a > >DB dump)? > >2) Would you trust the data after a programatic downgrade or would the data > >only really be trustworthy if from a restore? Specifically the new code > >*could* be relying on new data structures and >a downgrade could result in > >weird states of services. > > > >I’m looking at the expectation that a downgrade is possible. Each time I > >look at the downgrades I feel that it doesn’t make sense to ever really > >perform a downgrade outside of a development >environment. The potential for > >permanent loss of data / inconsistent data leads me to believe the downgrade > >is a flawed design. Input from the operators on real-world cases would be > >great to >have. > > > >This is an operator specific set of questions related to a post I made to > >the OpenStack development mailing list: > >http://lists.openstack.org/pipermail/openstack-dev/2015-January/055586.html > > > >Cheers, > >Morgan > > > > When moving major releases, we backup our databases and shutdown most of the > cluster so that portion of the cluster is still “good”. We then upgrade one > node completely, validate it, then join the rest of the nodes back in. If it > goes horribly wrong at that point we’d restore from backup. The main reasons > for me are two fold. First, rightly or wrongly, I assume that downgrades are > not well tested and rarely used by anyone. We certainly never test it during > our upgrade planning. Secondly, until I’m sure that the upgrade worked well, > I’d rather just go back to a clean state than rely on a downgrade just > because I know that state is 100% functional without further validation. > Especially if I’m in an outage window I don’t have time to mess around with a > downgrade and hope it works. I’ll kill the bad node and rebuild it, either > just restarting the old DB or restoring if needed. The tl;dr here is why take > the chance that the downgrade works when you have saner alternatives.
Yeah, this is the general approach we perform as well, and few other shops I know. > > > (please excuse the cruft below) > > > > > > This E-mail and any of its attachments may contain Time Warner Cable > proprietary information, which is privileged, confidential, or subject to > copyright belonging to Time Warner Cable. This E-mail is intended solely for > the use of the individual or entity to which it is addressed. If you are not > the intended recipient of this E-mail, you are hereby notified that any > dissemination, distribution, copying, or action taken in relation to the > contents of and attachments to this E-mail is strictly prohibited and may be > unlawful. If you have received this E-mail in error, please notify the sender > immediately and permanently delete the original and any copy of this E-mail > and any printout. > _______________________________________________ > OpenStack-operators mailing list > OpenStack-operators@lists.openstack.org > (mailto:OpenStack-operators@lists.openstack.org) > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > >
_______________________________________________ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
