The thread started with a discussion of problems with out of date databases. Mike pulled from master, rebuilt, and found out his testbed's database was out of date and no longer works.
We've tried to find a database solution, but out of databases are caused by more than schema changes. The data in the tables and other pieces of CloudStack can change after a commit, especially towards the end of release when everyone's feature is being merged. With 4.2, I dealt with this issue by writing a script that rebuilt my testbed by replaying the API calls I made to set it up in the first place. Is this a solution you've considered for your dev environment? DL > -----Original Message----- > From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] > Sent: 06 March 2014 12:50 > To: dev > Cc: Daan Hoogland > Subject: Re: [DISCUSS] Checking in code that will break others' environments > > No, but how do you mean with respect to this thread? > > On Thu, Mar 6, 2014 at 1:48 PM, Donal Lafferty <donal.laffe...@citrix.com> > wrote: > > Hi Daan, > > > > Is there anything stopping you from scripting the configuration of > > your CloudStack testbed? E.g. with Marvin or CloudMonkey > > > > DL > > > >> -----Original Message----- > >> From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] > >> Sent: 06 March 2014 09:53 > >> To: dev > >> Cc: Daan Hoogland > >> Subject: Re: [DISCUSS] Checking in code that will break others' > >> environments > >> > >> I totally agree with the incremental approach. I am a fascist at time > >> because i would even want people to add downgrade scripts to any db > >> change they make. Having them not adjust their sql is a good first step, > though. > >> > >> On Thu, Mar 6, 2014 at 10:43 AM, Miguel Ferreira > >> <mferre...@schubergphilis.com> wrote: > >> > Hi all, > >> > > >> > I've been looking at the way DB related code (SQL scripts and Java > >> > classes) > >> are updated and what is the impact of those updates on a live cloudstack > DB. > >> > By the way, my intention is to support a per-commit DB upgrade of a > >> running system. > >> > > >> > Anyway, I completely agree that sending an email warning people > >> > about > >> changes that can potentially break running environments is a good > >> thing, and can save a lot of time and headaches. > >> > I also like the idea of Rajani of using tools to help in the DB > >> > migration > >> process. > >> > However, I would like to put forward another idea: what about > >> > making sure > >> that the changes to the DB (schema and data) are always incremental? > >> > > >> > I mean, once a SQL statement (say A) is added to a script it could > >> > be kept as > >> is, and subsequent modifications cloud be made via new statements > >> (say B), instead of adapting A. > >> > This would allow people to upgrade their databases to the point > >> > they ran > >> statement A, and afterwards upgrade it again to the point they ran > >> statement B. > >> > The same principle could also be applied to the Java classes that > >> > do data > >> migration, but maybe there it might be a bit more involved. > >> > > >> > Cheers, > >> > Miguel > >> > > >> > -----Original Message----- > >> > From: Koushik Das [mailto:koushik....@citrix.com] > >> > Sent: donderdag 6 maart 2014 8:25 > >> > To: <dev@cloudstack.apache.org> > >> > Subject: Re: [DISCUSS] Checking in code that will break others' > >> > environments > >> > > >> > Before doing a git pull, I generally check the sql schema changes > >> > and run > >> the delta manually on my existing setup. In most of the cases that > >> works for me without having to redeploy the db. > >> > > >> > -Koushik > >> > > >> > On 06-Mar-2014, at 11:43 AM, Mike Tutkowski > >> <mike.tutkow...@solidfire.com> wrote: > >> > > >> >> Yeah, I definitely just meant a "heads up" during development if > >> >> you are going to change something that will break other people's > >> >> environments who update. If these people know in advance, they may > >> >> choose to postpone an update until they are at a better point. > >> >> > >> >> > >> >> On Wed, Mar 5, 2014 at 11:01 PM, Rajani Karuturi > >> >> <rajani.karut...@citrix.com > >> >>> wrote: > >> >> > >> >>> Across versions db migration is taken care. I think this is bound > >> >>> to occur while working on a release, if multiple people work on > >> >>> the same branch with different work-in-progress features. > >> >>> > >> >>> Could we move to flyway or liquibase which can take care of db > >> >>> versioning and migration? > >> >>> > >> >>> > >> >>> ~Rajani > >> >>> > >> >>> > >> >>> > >> >>> On 06-Mar-2014, at 2:08 am, Mike Tutkowski > >> >>> <mike.tutkow...@solidfire.com> > >> >>> wrote: > >> >>> > >> >>>> Yeah, in this case, I'm not referring to erroneous code that > >> >>>> breaks a person's environment (since hopefully the person > >> >>>> wouldn't have knowingly checked in such code), but rather, say, > >> >>>> DB-type changes that improve the system, but break current > setups. > >> >>>> > >> >>>> Just a heads-up e-mail with some easily identifiable tag. > >> >>>> > >> >>>> Can anyone think of a good tag for this? It's not always DB > >> >>>> related, so > >> >>> we > >> >>>> might want the tag to be more general. > >> >>>> > >> >>>> > >> >>>> On Wed, Mar 5, 2014 at 1:28 PM, Ian Duffy <i...@ianduffy.ie> > wrote: > >> >>>> > >> >>>>> +1 to this. > >> >>>>> > >> >>>>> Having the build suddenly break due to a git pull has been very > >> >>> annoying! > >> >>>>> I usually end up searching through the commit log and doing a > >> >>>>> resets until I find a commit where it works. Then waiting > >> >>>>> awhile until I do a git pull again and hoping the code was fixed. > >> >>>>> > >> >>>>> On 5 March 2014 20:19, Mike Tutkowski > >> >>>>> <mike.tutkow...@solidfire.com> > >> >>>>> wrote: > >> >>>>>> Hi, > >> >>>>>> > >> >>>>>> I encountered a bit of a problem this morning and thought I > >> >>>>>> would bring > >> >>>>> it > >> >>>>>> up for discussion. > >> >>>>>> > >> >>>>>> If we already have a policy around this, please let me know. > >> >>>>>> > >> >>>>>> So, I fetched the latest and rebased my local 4.4 development > >> >>>>>> branch on > >> >>>>> top > >> >>>>>> of master. This all went just fine. > >> >>>>>> > >> >>>>>> When I rebuilt and re-started the CS Management Server, I soon > >> >>> realized I > >> >>>>>> could no longer log in from the GUI. > >> >>>>>> > >> >>>>>> As it turns out, the DB schema had been updated and so my > >> >>>>>> database was > >> >>>>> out > >> >>>>>> of date. The code was querying for fields that didn't exist in my > DB. > >> >>>>>> > >> >>>>>> As far as I know, the easiest way to get around this is to > >> >>>>>> destroy my current cloud, run the script to re-build my > >> >>>>>> database, then re-create > >> >>> my > >> >>>>>> cloud, which is somewhat time consuming. > >> >>>>>> > >> >>>>>> Do we have a process in place currently in which we ask those > >> >>>>>> who make > >> >>>>> such > >> >>>>>> changes to send out a notification e-mail to dev@ to give > >> >>>>>> people a > >> >>>>> heads up > >> >>>>>> that updating will lead to such issues? On previous projects, > >> >>>>>> we would > >> >>>>> send > >> >>>>>> out an e-mail and then people could be aware to only update if > >> >>>>>> they > >> >>> were > >> >>>>>> prepared for such re-work. > >> >>>>>> > >> >>>>>> To be clear here, I'm not meaning to pick on anyone in > >> >>> particular...this > >> >>>>>> has happened several times over the course of my CloudStack > >> >>>>>> development > >> >>>>> and > >> >>>>>> I expect that I, too, have checked in such code (without > >> >>>>>> sending out a relevant e-mail) that lead people to have to > >> >>>>>> perform such a complete re-build action un-expectedly. > >> >>>>>> > >> >>>>>> What do people think about this? Maybe we should just add an > >> >>>>>> e-mail tag > >> >>>>> or > >> >>>>>> something and point people to the relevant commit? > >> >>>>>> > >> >>>>>> Thanks! > >> >>>>>> > >> >>>>>> -- > >> >>>>>> *Mike Tutkowski* > >> >>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >> >>>>>> e: mike.tutkow...@solidfire.com > >> >>>>>> o: 303.746.7302 > >> >>>>>> Advancing the way the world uses the > >> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play> > >> >>>>>> *(tm)* > >> >>>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> -- > >> >>>> *Mike Tutkowski* > >> >>>> *Senior CloudStack Developer, SolidFire Inc.* > >> >>>> e: mike.tutkow...@solidfire.com > >> >>>> o: 303.746.7302 > >> >>>> Advancing the way the world uses the > >> >>>> cloud<http://solidfire.com/solution/overview/?video=play> > >> >>>> *(tm)* > >> >>> > >> >>> > >> >> > >> >> > >> >> -- > >> >> *Mike Tutkowski* > >> >> *Senior CloudStack Developer, SolidFire Inc.* > >> >> e: mike.tutkow...@solidfire.com > >> >> o: 303.746.7302 > >> >> Advancing the way the world uses the > >> >> cloud<http://solidfire.com/solution/overview/?video=play> > >> >> *(tm)* > >> > > >> > >> > >> > >> -- > >> Daan > > > > -- > Daan
