From: Morgan Fainberg <morgan.fainb...@gmail.com<mailto:morgan.fainb...@gmail.com>> Date: Thursday, January 29, 2015 at 12:26 PM To: openstack-operators <openstack-operators@lists.openstack.org<mailto:openstack-operators@lists.openstack.org>> Subject: [Openstack-operators] [all] SQL Schema Downgrades: A Good Idea?
>From an operator perspective I wanted to get input on the SQL Schema >Downgrades. > >Today most projects (all?) provide a way to downgrade the SQL Schemas after >you’ve upgraded. Example would be moving from Juno to Kilo and then back to >Juno. There are some odd concepts >when handling a SQL migration downgrade >specifically around the state of the data. A downgrade, in many cases, causes >permanent and irrevocable data loss. When phrased like that (and >dusting off >my deployer/operator hat) I would be hesitant to run a downgrade in any >production, stagings, or even QA environment. > >In light of what a downgrade actually means I would like to get the views of >the operators on SQL Migration Downgrades: > >1) Would you actually perform a programatic downgrade via the cli tools or >would you just do a restore-to-last-known-good-before-upgrade (e.g. from a DB >dump)? >2) Would you trust the data after a programatic downgrade or would the data >only really be trustworthy if from a restore? Specifically the new code >*could* be relying on new data structures and >a downgrade could result in >weird states of services. > >I’m looking at the expectation that a downgrade is possible. Each time I look >at the downgrades I feel that it doesn’t make sense to ever really perform a >downgrade outside of a development >environment. The potential for permanent >loss of data / inconsistent data leads me to believe the downgrade is a flawed >design. Input from the operators on real-world cases would be great to >have. > >This is an operator specific set of questions related to a post I made to the >OpenStack development mailing list: >http://lists.openstack.org/pipermail/openstack-dev/2015-January/055586.html > >Cheers, >Morgan When moving major releases, we backup our databases and shutdown most of the cluster so that portion of the cluster is still “good”. We then upgrade one node completely, validate it, then join the rest of the nodes back in. If it goes horribly wrong at that point we’d restore from backup. The main reasons for me are two fold. First, rightly or wrongly, I assume that downgrades are not well tested and rarely used by anyone. We certainly never test it during our upgrade planning. Secondly, until I’m sure that the upgrade worked well, I’d rather just go back to a clean state than rely on a downgrade just because I know that state is 100% functional without further validation. Especially if I’m in an outage window I don’t have time to mess around with a downgrade and hope it works. I’ll kill the bad node and rebuild it, either just restarting the old DB or restoring if needed. The tl;dr here is why take the chance that the downgrade works when you have saner alternatives. (please excuse the cruft below) ________________________________ This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.
_______________________________________________ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
