Re: Let’s discuss database upgrades

2016-01-11 Thread Rafael Weingärtner
nowledge will decay pretty > >>>> quickly. > >>>> There is a need for an upgrade that works reliably and has good tests > >>>> that > >>>> can be quickly tried to see that the upgrade has worked or needs to be > >>>> reverted. > &

Re: Let’s discuss database upgrades

2016-01-03 Thread John Burwell
it to be smooth and we absolutely need >>>>> it >>>>> to be outside of the main repo and able to rollback if stuff goes wrong >>>>> (so >>>>> users can retry). >>>>> >>>>> The biggest other issue I see in upgrading is

Re: Let’s discuss database upgrades

2016-01-03 Thread Rafael Weingärtner
er issue I see in upgrading is the systemvm replacement >>>> and having to reboot (100s or 1000s of routers). That’s where your real >>>> downtime is most of the time. >>>> >>>> If you have done all that and have to revert, it is not very comforting >>> to

Re: Let’s discuss database upgrades

2016-01-03 Thread Ron Wheeler
t;" < dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>> Subject: RE: Let’s discuss database upgrades Hi Guys, from the user's perspective, there are two points which come up again and again - 1. lack a prescribed roll back if an upgrade goes badly 2. The downtime involved

Re: Let’s discuss database upgrades

2016-01-03 Thread Rafael Weingärtner
; > >> Regards, >> Remi >> >> >> From: Paul Angus > paul.an...@shapeblue.com>> >> Reply-To: "dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>" < >> dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>&g

Re: Let’s discuss database upgrades

2015-12-30 Thread Ron Wheeler
<mailto:dev@cloudstack.apache.org>" mailto:dev@cloudstack.apache.org>> Date: Wednesday 30 December 2015 10:10 To: "dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>" mailto:dev@cloudstack.apache.org>> Subject: RE: Let’s discuss database upgrades Hi Guys,

Re: Let’s discuss database upgrades

2015-12-30 Thread Remi Bergsma
g>> Date: Wednesday 30 December 2015 10:10 To: "dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>" mailto:dev@cloudstack.apache.org>> Subject: RE: Let’s discuss database upgrades Hi Guys, from the user's perspective, there are two points which come up a

RE: Let’s discuss database upgrades

2015-12-30 Thread Paul Angus
il.com] Sent: 29 December 2015 21:45 To: dev Subject: Re: Let’s discuss database upgrades On Mon, Dec 28, 2015 at 2:16 PM, Rafael Weingärtner < rafaelweingart...@gmail.com> wrote: > Hi all devs, > First of all, sorry the long text, but I hope we can start a > discussion here and imp

Re: Let’s discuss database upgrades

2015-12-29 Thread Erik Weber
On Mon, Dec 28, 2015 at 2:16 PM, Rafael Weingärtner < rafaelweingart...@gmail.com> wrote: > Hi all devs, > First of all, sorry the long text, but I hope we can start a discussion > here and improve that part of ACS. > > A while ago I have faced the code that Apache CloudStack (ACS) uses to > upgra

Re: Let’s discuss database upgrades

2015-12-29 Thread Ron Wheeler
On 29/12/2015 12:01 PM, Daan Hoogland wrote: You are right Ron, but there is always security fixes that might require db changes on a point release. That might not be so bad if a) they are very exceptional b) the bug is so critical that all users of version 4.5.x and earlier will have to move t

Re: Let’s discuss database upgrades

2015-12-29 Thread Daan Hoogland
You are right Ron, but there is always security fixes that might require db changes on a point release. On Tue, Dec 29, 2015 at 5:39 PM, Ron Wheeler wrote: > As an old-timer but a new cloudstack user, it strikes me as a bit odd > that changes to the database are allowed within a minor version c

Re: Let’s discuss database upgrades

2015-12-29 Thread Ron Wheeler
As an old-timer but a new cloudstack user, it strikes me as a bit odd that changes to the database are allowed within a minor version change. This seems to cause a lot more problems than it solves. It could delay the release of someone's pet enhancement or bug fix but the idea of not being abl

Re: Let’s discuss database upgrades

2015-12-29 Thread Daan Hoogland
CCYY == On Tue, Dec 29, 2015 at 3:06 PM, Rafael Weingärtner < rafaelweingart...@gmail.com> wrote: > I also liked the date-format, what did you mean with CCYY? > > > > The way I think we might have a problem, would be to commits/PRs that end > up creating files with same names. Then, we would

Re: Let’s discuss database upgrades

2015-12-29 Thread Rafael Weingärtner
I also liked the date-format, what did you mean with CCYY? The way I think we might have a problem, would be to commits/PRs that end up creating files with same names. Then, we would have to agree upon a way to solve those conflicts, such as appending an extra character to indicate a sequence to

Re: Let’s discuss database upgrades

2015-12-29 Thread Wido den Hollander
On 29-12-15 14:46, Daan Hoogland wrote: > Wido, Rafael, > > I like the date-format but then of course CCYY-MM-DD. I can still think of > ways to screw up that (or the plain int;) > 20151229 is a valid integer which you can simply use to compare with. 100, 101, 102 or 20151229, 20160103, 20160

Re: Let’s discuss database upgrades

2015-12-29 Thread Daan Hoogland
By this I meant I'm fine with any decision the implementer makes, btw. We will encounter the administrative issues with it and deal with them. On Tue, Dec 29, 2015 at 2:46 PM, Daan Hoogland wrote: > Wido, Rafael, > > I like the date-format but then of course CCYY-MM-DD. I can still think of > wa

Re: Let’s discuss database upgrades

2015-12-29 Thread Daan Hoogland
Wido, Rafael, I like the date-format but then of course CCYY-MM-DD. I can still think of ways to screw up that (or the plain int;) On Tue, Dec 29, 2015 at 1:40 PM, Rafael Weingärtner < rafaelweingart...@gmail.com> wrote: > Wido, that is true, you are right; the naming on upgrade routines can use

Re: Let’s discuss database upgrades

2015-12-29 Thread Rafael Weingärtner
Wido, that is true, you are right; the naming on upgrade routines can use a numeric value independent of the number of the version. The numeric value can be a simple integer that is incremented each routine that is added or a time stamp when the routine was added. The point is that we would have to

Re: Let’s discuss database upgrades

2015-12-29 Thread Wido den Hollander
On 29-12-15 13:21, Rafael Weingärtner wrote: > I got your point Daan. > > Well, and if we linked a version of ACS with a time stamp in the format of > DD.MM.? > In that case you could also say. ACS 4.6.0 == db ver X You don't have to say ver >= X, you can also say ver = X. > We could th

Re: Let’s discuss database upgrades

2015-12-29 Thread Rafael Weingärtner
I got your point Daan. Well, and if we linked a version of ACS with a time stamp in the format of DD.MM.? We could then use the time stamp in the same format to name upgrade routines. This way the idea of running all of the routines in between version during upgrades could be applied. On Tue

Re: Let’s discuss database upgrades

2015-12-29 Thread Daan Hoogland
Rafael, On Tue, Dec 29, 2015 at 12:22 PM, Rafael Weingärtner < rafaelweingart...@gmail.com> wrote: > Thanks, Daan and Wido for your contributions, I will discuss them as > follows. > > Daan, about the idea of per commit upgrades. Do you mean that we separate > each change in the database that is

Re: Let’s discuss database upgrades

2015-12-29 Thread Rafael Weingärtner
Thanks, Daan and Wido for your contributions, I will discuss them as follows. Daan, about the idea of per commit upgrades. Do you mean that we separate each change in the database that is introduced by PRs/Commits in a different file (routine upgrade) per ACS version? So we would have, V_480_A.sql

Re: Let’s discuss database upgrades

2015-12-29 Thread Wido den Hollander
On 28-12-15 21:34, Rafael Weingärtner wrote: > Hi Wido, Rohit, > I have just read the feature suggestion. > > Wido, I am not trying to complicate things, quite the opposite, I just > illustrate a simple thing that can happen and is happening; I just pointed > how it can be easily solved. > > Ab

Re: Let’s discuss database upgrades

2015-12-29 Thread Daan Hoogland
€0,02: I think it is important to support per commit upgrades, so I not with Wido on this at all. Looking at your naming scheme that might be a problem. Other then this your initial proposal seems fine. The reason is not to allow for x.y.z schema changes but to be able to decide a certain commit g

Re: Let’s discuss database upgrades

2015-12-28 Thread Rafael Weingärtner
Hi Wido, Rohit, I have just read the feature suggestion. Wido, I am not trying to complicate things, quite the opposite, I just illustrate a simple thing that can happen and is happening; I just pointed how it can be easily solved. About the release of .Z, releases more constant and others, I do

Re: Let’s discuss database upgrades

2015-12-28 Thread Rohit Yadav
Hi Rafael and Wido, Thanks for starting a conversation in this regard, I could not pursue the Chimp tool due to other $dayjob work though it’s good to see some discussion has started again. Hope we’ll solve this in 2016. In my opinion, we will need to first separate the database init/migration

Re: Let’s discuss database upgrades

2015-12-28 Thread Wido den Hollander
On 28-12-15 16:21, Rafael Weingärtner wrote: > Thanks for your contribution Wido, > I have not seen Rohit’s email; I will take a look at it. > Ok, he has a FS here: https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+Chimp > About database schema changes happening only in X.Y, I

Re: Let’s discuss database upgrades

2015-12-28 Thread Rafael Weingärtner
Thanks for your contribution Wido, I have not seen Rohit’s email; I will take a look at it. About database schema changes happening only in X.Y, I also agree with you (that is a convention we all could agree on, and such as conding and release procedures we could have a wiki page for that). Howeve

Re: Let’s discuss database upgrades

2015-12-28 Thread Wido den Hollander
On 28-12-15 14:16, Rafael Weingärtner wrote: > Hi all devs, > First of all, sorry the long text, but I hope we can start a discussion > here and improve that part of ACS. > > A while ago I have faced the code that Apache CloudStack (ACS) uses to > upgrade from a version to newer one and that did