> -----Original Message----- > From: Erik Weber [mailto:terbol...@gmail.com] > Sent: Wednesday, August 20, 2014 12:25 PM > To: dev > Subject: Re: [PROPOSAL] Using continuous integration to maintain our code > quality... > > On Wed, May 7, 2014 at 2:03 AM, Alex Huang <alex.hu...@citrix.com> > wrote: > > > Hi All, > > > > This is something I brought up a long time ago but really didn't have > > the resources to get it all up and running until now. Throughout the > > past year, Edison, Prasanna, Amogh, Bharat, Koushik, Talluri, and > > others have been chipping away at it. At this point, we finally pull > > together a continuous integration setup that we can use to make sure > > that CloudStack master and the currently release branch are always > > stable. This is getting pretty close to be completed and we like to > > share it with the community in hopes that we can reduce/eliminate that > > problems we've seen with our recent releases. Currently, the physical > > hardware are hosted by Citrix but we'll be more than willing to donate the > work to infra when that's all settled. > > > > This does require effort from the community to make a change in their > > development process. These steps are detailed at [1]. I like to get > > feedback on what everyone think about this. > > > > What have we done: > > - We replaced a large selection of the BVT tests to run with the > > simulator instead of actual hardware. This shortens the duration of > > each BVT run. Today, a BVT that runs tests for XenServer and KVM > > completes in > > 30-40 minutes. > > > > How much is running with Simulator instead of actual hardware? My issue > with this is that you're testing against a flawless simulator in terms of > testing, > while with actual hardware you are bound to hit bugs/issues that might not > be due to ACS code but ACS still has to handle it. > > As an example, could you run a test on the tags '4.4.0' and '4.3.0' and report > the result? They both had fundamental flaws, where the one was practically > useless for a week or so, and the other had major issues with KVM, and if the > BVT doesn't encounter those because it's using the simulator I see it as a > burden rather than a gift, since you're relying on a false result.
For KVM, there is another way to test basic KVM features without using real HW: https://cwiki.apache.org/confluence/display/CLOUDSTACK/devcloud-kvm Currently, I am working on a machine I created in digtialOcean, and make sure it can test against real KVM, not only just simulator. If it's working, I can donate it as one of CI machine. > > -- > Erik
