John, Usually testing of templates require regression testing which usually take ~2 week ( need to run core regression on them ). It is usually preferred to test templates early on in release given the risk and also it would get enough coverage by all the testing that happens during release cycles.
We just got the 4.2 templates working couple of weeks ago and some additional changes are being made for RVR related fixes - I think Bharat has made some change yesterday. There are features closed in 4.2 early on and QA used older templates for their testing. Those features will be using new templates in second cycle run. I would say those templates are tested 80% in 4.2. Using the 4.2 templates for 4.1 code base might not be that trivial in my opinion. Thanks /Sudha -----Original Message----- From: John Burwell [mailto:jburw...@basho.com] Sent: Tuesday, May 21, 2013 5:35 PM To: dev@cloudstack.apache.org Subject: Re: [DISCUSS] Should we pause merges into master until 4.1 is out the door? All, Can someone please answer the following questions: 1. Besides testing, what work needs to be done back port the 4.2 system VMs to 4.1 (e.g. docs, posting images for download, etc)? 2. What is involved to test/verify the operation of 4.2 system VMs on 4.1? What is labor/time estimate? 3. How can community members help accelerate the work in 1 and 2? Thanks, -John On May 21, 2013, at 8:24 PM, Chiradeep Vittal <chiradeep.vit...@citrix.com> wrote: > I'm not arguing the value of blockers. Sure they are valuable. But our > estimate of the harm (IMO) is totally out of proportion. Take the time > sync issue. This has been there since 2.2.0. Are we saying that the > thousands of production Cloudstack clouds are well and truly borked > and cannot function? Nope. They work just fine. Should we fix it? Yes, > but to Dave's point, we cannot hold up this release any longer. > > > On 5/21/13 5:11 PM, "John Burwell" <jburw...@basho.com> wrote: > >> Chiradeep, >> >> This defect affects 100% of users that use system VMs which I believe >> is all of them. It also appears that we have a fix for this problem >> that needs to be pulled back from 4.2 and tested. What is involved >> with testing it? >> >> Personally, I would be please if we found more blockers before the >> release of 4.1.0. The ideal is that the quality of our .0 releases >> does not require subsequent patch release. While such an ideal is >> not possible, it should be goal to which we strive. As has been >> pointed out in the 4.0.3 discussion, each patch release has a cost. >> My hope is that 4.1.0 is of high enough quality that any defect fixes >> can be held to 4.2.0. >> >> Thanks, >> -John >> >> On May 21, 2013, at 8:00 PM, Chiradeep Vittal >> <chiradeep.vit...@citrix.com> wrote: >> >>> But the longer we hold the window open for folks to raise defects, >>> the longer it will take to release. Why can't we enforce our own >>> timelines and say "this is it". Any release will have blockers for a >>> subset of users. >>> It >>> seems to me that we are inefficient in estimating the harm from a >>> 'blocker' defect -- I.e., the defect is assumed to affect 100% of >>> the users and therefore blocks the release. There's always 4.1.1 >>> >>> >>> On 5/21/13 2:20 PM, "David Nalley" <da...@gnsa.us> wrote: >>> >>>> On Tue, May 21, 2013 at 4:18 PM, Chip Childers >>>> <chip.child...@sungard.com> wrote: >>>>> On Tue, May 21, 2013 at 12:34:45AM +0000, Chiradeep Vittal wrote: >>>>>> I don't see limited interest. It seems that bugs are trickling in >>>>>> every day and they are being taken up as they come in. Is there >>>>>> any blocker without any action for more than a few days? The only >>>>>> one I can see CLOUDSTACK-2463. >>>>> >>>>> Chiradeep, >>>>> >>>>> My response to Animesh was flippant and not overly helpful. You >>>>> are correct that things are being addressed. My point was more >>>>> that the community in general seems to have moved on from 4.1, yet >>>>> we have not released it yet. Bugs that have come up are taking >>>>> several requests for attention, and once there is a reply it's >>>>> frequently taking several requests to get follow ups. This is a >>>>> volunteer project, so that alone isn't the issue. I raise the >>>>> question about what to do about 4.1 in the interest of asking the >>>>> rest of the community if you have, indeed, moved on and want to >>>>> focus on 4.2 instead. >>>>> >>>>> Others, >>>>> >>>>> I'm frankly surprised at how few people have responded to this >>>>> thread, given the volume of commit / merge / jira activity going >>>>> on for new features. >>>>> Obviously there is lots of effort going into new feature dev, so >>>>> it's not at all like we have stopped paying attention to the >>>>> project as a community (far from it). >>>>> >>>>> As for the current state of consensus around my questions: >>>>> >>>>> * Animesh indicated a desire to keep moving on both fronts. >>>>> * Prasanna indicated his concern that changes in master are being >>>>> missed by people looking at 4.1. >>>>> * John indicated his concern about the priority conflict WRT >>>>> stabilizing >>>>> 4.2 and 4.1 concurrently. >>>>> * Chiradeep - I know you replied to this thread (obviously), but >>>>> I'm not sure if I saw an answer to the questions I raised >>>>> (although you make a fair point, which I address above). >>>>> >>>>> I'm looking for more feedback one way or the other here. >>>>> >>>>> -chip >>>> >>>> >>>> So here's my thoughts - and I apologize if this seems like a rant. >>>> >>>> The goal of the project is to release code. It's the cornerstone of >>>> much of what we do. We've done a very good job in my opinion of >>>> making CloudStack consumable by people who are comfortable hacking >>>> on the source code. We have a number of people running versions of >>>> CloudStack that have never been released yet, and doing so pretty >>>> confidently. >>>> Most of our target audience is either not comfortable doing that, >>>> or not comfortable running something in production that hasn't been >>>> blessed as a release, and doesn't have a known upgrade path. >>>> >>>> While it's not just the goal and purpose of the project to release >>>> code - it's also vital to the health and growth of our user community. >>>> Regular, timely releases are important. The 50,000 foot view of >>>> things is that there is apathy about the 4.1 release. Lots of >>>> activity is happening around feature development, but not a lot of >>>> care (even in form of opinions in these threads) given to some of >>>> the 4.1 blocker issues. >>>> >>>> Performing a release is how we show the world how awesome we are, >>>> and how awesome our software is. Writing the software, developing >>>> cool new features and never pushing it out the door is a waste - >>>> virtually no one but us will see it. The equivalent of getting >>>> dressed up for a night on the town, but never leaving the house. In >>>> short it isn't done until there is a release, and seeing large >>>> features being developed and landing while bugs that block a >>>> release take a lot of coaxing to get fixed gives a bad impression. >>>> >>>> --David >
