I recently had an urgent family matter to take care and thanks for Alex to help me out on this matter. I think at this moment, without a previous 100% rate of BVT past rate, it will be hard to tell a direct link between a particular change with a BVT failure. It is a good idea to resolve this measurement tool issue first.
Kelven On 6/27/13 4:50 PM, "Alex Huang" <alex.hu...@citrix.com> wrote: >Dave, > >Chip has asked this before and we also stated specifically that we won't >merge it in unless we see equivalent pass rate on the BVT as master. >We're doing that right now. > >--Alex > >> -----Original Message----- >> From: David Nalley [mailto:da...@gnsa.us] >> Sent: Thursday, June 27, 2013 4:34 PM >> To: dev@cloudstack.apache.org >> Subject: Re: [MERGE] Merge VMSync improvement branch into master >> >> On Thu, Jun 27, 2013 at 5:51 PM, Hugo Trippaers <h...@trippaers.nl> >>wrote: >> > >> > I think Ilya offers is great, my current stance is also to see how we >>can bring >> this forward. >> > >> > I've had the opportunity to meet with several people at the Citrix >>office in >> Santa Clara, i'm actually working from their office at this moment. I >>think it's >> also the responsibility of someone who put in a -1 to work with the >>original >> committer to get the situation resolved. So i'll invest the time to >>help with >> the review as well. >> > >> > It would be great if Alex or Kelven could take the time to explain >>how this >> feature has been tested. That would give the community some insight as >> well. >> > >> > My main technical problem with this merge is that stuff is moving all >>over >> the place without having even the slightest idea why. Now having >>discussed >> this with Alex in person i get the general idea of this merge, so can >>actually >> try to review it. >> > >> > I think that John have nicely explained what we could do to prevent >> situations like this in advance. I fully understand that big features >>or rewrites >> don't happen overnight and might show up near the end of the release >>cycle. >> With the time based release cycle it's always a risk that some feature >>might >> not make it in on time. Getting more people involved and chunking the >> commits into master will greatly speed up the reviewing process. >> > >> > I'll get back to this after spending some time on reviewing the >>actual patch. >> In fact i would like to ask more people to have a look at this patch >>and reply >> to this thread with comments or remarks. >> > >> > Cheers, >> > >> > Hugo >> > >> >> So the problem in my mind, is that we don't have a way of verifying that >> master isn't broken, and won't be broken by any given merge. I look at >>even >> the minimal level of automated testing that I see today, and ~20% of >> integration tests are failing[1]. The regression set of tests (which >>isn't running >> as often) is seeing 75% of tests failing[2]. Heaping on more change >>when we >> are demonstrably already failing in many places is not behaving >>responsibly >> IMO. >> The question I'd pose is this - running the various automated tests is >>pretty >> cheap - whats the output of that compared to the current test output on >> master? Better or worse? If it hasn't been done, why not? >> I desperately want these features, but not necessarily at the cost of >>further >> destabilizing what we have now in master - we can't continue accruing >> technical debt. >> >> --David >> >> [1] http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-smoke- >> matrix/lastCompletedBuild/testReport/ >> [2] >>http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-regression- >> matrix/28/testReport/
