Re: Let’s discuss database upgrades

2016-01-11 Thread Rafael Weingärtner
system VMs. Consider the number of maintenance windows AWS opens where all or part of its services are completely unavailable. Therefore, our tooling needs to favor additive, non-destructive schema changes that allow database upgrades to be applied while an older version of the management is still

Re: Let’s discuss database upgrades

2016-01-03 Thread John Burwell
onerous. Consider the number of maintenance windows AWS opens where all or part of its services are completely unavailable. Therefore, our tooling needs to favor additive, non-destructive schema changes that allow database upgrades to be applied while an older version of the management is still

Re: Let’s discuss database upgrades

2016-01-03 Thread Rafael Weingärtner
er issue I see in upgrading is the systemvm replacement >>>> and having to reboot (100s or 1000s of routers). That’s where your real >>>> downtime is most of the time. >>>> >>>> If you have done all that and have to revert, it is not very comforting >>> to

Re: Let’s discuss database upgrades

2016-01-03 Thread Ron Wheeler
t;" < dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>> Subject: RE: Let’s discuss database upgrades Hi Guys, from the user's perspective, there are two points which come up again and again - 1. lack a prescribed roll back if an upgrade goes badly 2. The downtime involved

Re: Let’s discuss database upgrades

2016-01-03 Thread Rafael Weingärtner
; > >> Regards, >> Remi >> >> >> From: Paul Angus > paul.an...@shapeblue.com>> >> Reply-To: "dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>" < >> dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>&g

Re: Let’s discuss database upgrades

2015-12-30 Thread Ron Wheeler
<mailto:dev@cloudstack.apache.org>" mailto:dev@cloudstack.apache.org>> Date: Wednesday 30 December 2015 10:10 To: "dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>" mailto:dev@cloudstack.apache.org>> Subject: RE: Let’s discuss database upgrades Hi Guys,

Re: Let’s discuss database upgrades

2015-12-30 Thread Remi Bergsma
g>> Date: Wednesday 30 December 2015 10:10 To: "dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>" mailto:dev@cloudstack.apache.org>> Subject: RE: Let’s discuss database upgrades Hi Guys, from the user's perspective, there are two points which come up a

RE: Let’s discuss database upgrades

2015-12-30 Thread Paul Angus
il.com] Sent: 29 December 2015 21:45 To: dev Subject: Re: Let’s discuss database upgrades On Mon, Dec 28, 2015 at 2:16 PM, Rafael Weingärtner < rafaelweingart...@gmail.com> wrote: > Hi all devs, > First of all, sorry the long text, but I hope we can start a > discussion here and imp

Re: Let’s discuss database upgrades

2015-12-29 Thread Erik Weber
On Mon, Dec 28, 2015 at 2:16 PM, Rafael Weingärtner < rafaelweingart...@gmail.com> wrote: > Hi all devs, > First of all, sorry the long text, but I hope we can start a discussion > here and improve that part of ACS. > > A while ago I have faced the code that Apache CloudStack (ACS) uses to > upgra

Re: Let’s discuss database upgrades

2015-12-29 Thread Ron Wheeler
the release of .Z, releases more constant and others, I do not want to mix topics. Let’s keep this thread strict to discuss database upgrades. I do not want to start the release discussion, but what I meant is that we try to find a technical solution to something which might be solved

Re: Let’s discuss database upgrades

2015-12-29 Thread Daan Hoogland
t;>> >>>>>>>>> >>>>>>>>> Hi Wido, now I understand your example. >>>>>>>>>> I understand your worry about upgrade paths, and that is the >>>>>>>>>> >>>>>>>>> point

Re: Let’s discuss database upgrades

2015-12-29 Thread Ron Wheeler
rate a simple thing that can happen and is happening; I just pointed how it can be easily solved. About the release of .Z, releases more constant and others, I do not want to mix topics. Let’s keep this thread strict to discuss database upgrades. I do not want to start the release discussion, bu

Re: Let’s discuss database upgrades

2015-12-29 Thread Daan Hoogland
;>>>>> should be able to run all of the upgrade routines in between them > > >>>>>> (including the upgrade routine of the goal version). Therefore, if > > we > > >>>>>> release a version 4.6.0, and then 4.5.3, if someone upgrades to > > 4.5.3 > &g

Re: Let’s discuss database upgrades

2015-12-29 Thread Rafael Weingärtner
;>>>> any other version, and then wants to upgrade to 4.6.0, that would > not > >>> be > >>>>> a > >>>>>> problem, it would be a metter of running only the routine upgrade of > >>>>> 4.6.0 > >>>>>>

Re: Let’s discuss database upgrades

2015-12-29 Thread Wido den Hollander
t; implicit by our upgrade conventions. >>>>>> >>>>>> About creating versions of the code that rely on some version of the >>>>>> database. I do not like much because of compatibility issues that >> might >>>>>> arise. For inst

Re: Let’s discuss database upgrades

2015-12-29 Thread Daan Hoogland
gt; > >>> version. We do not need to explicitly create upgrade paths. They >> should >> > >> be >> > >>> implicit by our upgrade conventions. >> > >>> >> > >>> About creating versions of the code

Re: Let’s discuss database upgrades

2015-12-29 Thread Daan Hoogland
instance, let’s say version X of ACS depends on version > >=Y > > of > > >>> the database. If I upgrade the database to version Y + 1 or +2, the > > same > > >>> ACS version has to keep running nice and shiny. My worry is that may > > >> bring

Re: Let’s discuss database upgrades

2015-12-29 Thread Rafael Weingärtner
y. My worry is that may > >> bring > >>> some complications, such as to remove columns that cease to be used or > >> data > >>> structure that we might want to improve. > >>> > >>> I normally see that the database version and the code base

Re: Let’s discuss database upgrades

2015-12-29 Thread Wido den Hollander
hat the database version and the code base are tied in a >>> mapping 1 to 1. Maybe I am having troubles identifying the benefits of >> that >>> change. >>> >>> Thanks for your time ;) >>> >>> On Tue, Dec 29, 2015 at 8:15 AM, Wido den Holla

Re: Let’s discuss database upgrades

2015-12-29 Thread Rafael Weingärtner
5 AM, Wido den Hollander > > wrote: > > > > > > > > > > > On 28-12-15 21:34, Rafael Weingärtner wrote: > > > > Hi Wido, Rohit, > > > > I have just read the feature suggestion. > > > > > > > > Wido, I am not try

Re: Let’s discuss database upgrades

2015-12-29 Thread Daan Hoogland
omebody > > upgrade from 4.5.3 to 4.6.0? He can't, since the 4.6.0 code doesn't > > support that path. > > > > So my idea is to split the database version from the code version. > > > > The code requires database version >= X and during boot it

Re: Let’s discuss database upgrades

2015-12-29 Thread Rafael Weingärtner
thing that can happen and is happening; I just > pointed > > how it can be easily solved. > > > > About the release of .Z, releases more constant and others, I do not want > > to mix topics. Let’s keep this thread strict to discuss database > upgrades. > > > >

Re: Let’s discuss database upgrades

2015-12-29 Thread Wido den Hollander
n be easily solved. > > About the release of .Z, releases more constant and others, I do not want > to mix topics. Let’s keep this thread strict to discuss database upgrades. > I do not want to start the release discussion, but what I meant is that we try to find a technical solution to s

Re: Let’s discuss database upgrades

2015-12-29 Thread Daan Hoogland
ing that can happen and is happening; I just pointed > how it can be easily solved. > > About the release of .Z, releases more constant and others, I do not want > to mix topics. Let’s keep this thread strict to discuss database upgrades. > > Now, about the FS. I agree with Rohit th

Re: Let’s discuss database upgrades

2015-12-28 Thread Rafael Weingärtner
not want to mix topics. Let’s keep this thread strict to discuss database upgrades. Now, about the FS. I agree with Rohit that we should have only one way of managing database upgrades and creation. I just do not like the idea of creating a tool that work as a wrapper on frameworks/tools such as

Re: Let’s discuss database upgrades

2015-12-28 Thread Rohit Yadav
Hi Rafael and Wido, Thanks for starting a conversation in this regard, I could not pursue the Chimp tool due to other $dayjob work though it’s good to see some discussion has started again. Hope we’ll solve this in 2016. In my opinion, we will need to first separate the database init/migration

Re: Let’s discuss database upgrades

2015-12-28 Thread Wido den Hollander
On 28-12-15 16:21, Rafael Weingärtner wrote: > Thanks for your contribution Wido, > I have not seen Rohit’s email; I will take a look at it. > Ok, he has a FS here: https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+Chimp > About database schema changes happening only in X.Y, I

Re: Let’s discuss database upgrades

2015-12-28 Thread Rafael Weingärtner
Thanks for your contribution Wido, I have not seen Rohit’s email; I will take a look at it. About database schema changes happening only in X.Y, I also agree with you (that is a convention we all could agree on, and such as conding and release procedures we could have a wiki page for that). Howeve

Re: Let’s discuss database upgrades

2015-12-28 Thread Wido den Hollander
On 28-12-15 14:16, Rafael Weingärtner wrote: > Hi all devs, > First of all, sorry the long text, but I hope we can start a discussion > here and improve that part of ACS. > > A while ago I have faced the code that Apache CloudStack (ACS) uses to > upgrade from a version to newer one and that did

Let’s discuss database upgrades

2015-12-28 Thread Rafael Weingärtner
Hi all devs, First of all, sorry the long text, but I hope we can start a discussion here and improve that part of ACS. A while ago I have faced the code that Apache CloudStack (ACS) uses to upgrade from a version to newer one and that did not seem to be a good way to execute our upgrades. Therefo

RE: database upgrades

2013-08-28 Thread Alex Huang
sage- > From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com] > Sent: Wednesday, August 28, 2013 9:20 AM > To: dev@cloudstack.apache.org > Cc: dev@cloudstack.apache.org > Subject: Re: database upgrades > > Having the ability to do DB migrations on startup isn't bad.

Re: database upgrades

2013-08-28 Thread Darren Shepherd
From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com] >> Sent: Tuesday, August 27, 2013 4:48 PM >> To: dev@cloudstack.apache.org >> Subject: database upgrades >> >> It seems like on startup of the management server >> DatabaseUpgradeChecker runs and upgrad

RE: database upgrades

2013-08-28 Thread Alex Huang
--- > From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com] > Sent: Tuesday, August 27, 2013 4:48 PM > To: dev@cloudstack.apache.org > Subject: database upgrades > > It seems like on startup of the management server > DatabaseUpgradeChecker runs and upgrades the database perfo

Re: database upgrades

2013-08-27 Thread Prasanna Santhanam
The whole upgrade process can probably be divided into a few parts: 1. Apply new schema 2. Data Migration (this is the stored proc updates if you will) 3. Upgrade of system scripts and templates 4. Phased service (agent/management) shutdown and java updates. As a first step if 1. and 2. can be de

Re: database upgrades

2013-08-27 Thread Darren Shepherd
Custom java code to manage the database is every DBAs worse nightmare. I'd be very very interested into moving to an off the shelf DB migration tool. I'd recommend flyway given the fact that CloudStack only supports mysql and is very SQL DDL oriented as it stands. I'll probably look into this

Re: database upgrades

2013-08-27 Thread Prasanna Santhanam
On Tue, Aug 27, 2013 at 04:48:02PM -0700, Darren Shepherd wrote: > It seems like on startup of the management server > DatabaseUpgradeChecker runs and upgrades the database performing DDL > and DML. Is there a way to upgrade the database that doesn't > include starting the management server? > T

database upgrades

2013-08-27 Thread Darren Shepherd
It seems like on startup of the management server DatabaseUpgradeChecker runs and upgrades the database performing DDL and DML. Is there a way to upgrade the database that doesn't include starting the management server? Darren