> -----Original Message----- > From: Chip Childers [mailto:chip.child...@sungard.com] > Sent: Monday, April 15, 2013 7:22 AM > To: dev@cloudstack.apache.org > Cc: cloudstack-...@incubator.apache.org > Subject: Re: [ASFCS42] Proposed schedule for our next release > > On Thu, Apr 11, 2013 at 02:50:02PM -0700, Animesh Chaturvedi wrote: > > > > I want to call out my concern on technical debt we have accumulated so > far. > > > > I did an analysis on JIRA bugs yesterday night PST on "Affects > > Version = 4.1" and created since Dec 2012 > > > > Total records : 429 > > Resolution Type (Invalid, Duplicate, Cannot reproduce etc.) : 87 (30 > > Blockers, 27 Critical, 27 Major, 4 Minor) Valid Defects : 429-87= 342 > > Fixed : 246 (60 Blockers, 70 Critical, 99 Majors) out of which 217 > > were fixed since Feb Unresolved : 96 (1 Blocker, 8 Critical, 64 Major) > > > > With this data it looks like we have fixed 2/3 of valid defects in little > > over > 2 months and pretty much deferring around 1/3 rd of issues for future > release. > > > > I also looked at overall backlog of bugs (Critical, Major and Blockers only) > as of 4/10/2013 - 10:0PM PST. > > > > 284 open (18 Blocker, 38 Critical, 228 Major) ; By Fix version > > - Release 4.0.x and prior: 13 > > - 4.1: 70 > > - 4.2 : 97 > > - Future: 8 > > - No version: 107 > > > > Looking at that we fixed 217 bugs in roughly 2 months during 4.1 cycle, > fixing the backlog of bug will probably take us 2 months. Should we extend > the 4.2 test cycle by 2 months [Original Schedule: 6/1 - 7/22, Extended > Schedule: 6/1-9/22] to reduce the technical debt significantly? I would like > to hear how community wants to address technical debt. Based on the > input and consensus I will publish the agreed schedule next week. > > > > > > I don't think that an extension of time changes bug counts really. IMO, we > need to pull together to have some bug-fix focused effort applied to the > code-base. It's also another reason that I'm so big on making sure that > automated tests come in with the new features. That doesn't address test > scenarios that human testers can come up with, but if a developer spends > the time to think about testing the basic feature and codifies that, we > should at least avoid the "this actually doesn't work at all" types of bugs. > > There's a school of thought that says, don't build another feature until you > have sorted out the known bugs in the current features. I don't think we > could really pull that off, but perhaps a different thread to rally people > around the bug backlog is in order? > > -chip
Sorry to chime in so late to this thread as I've been offsite for the better part of this week. I was one of the original 4 month release crowd but after the recent two releases of ACS, I'm starting to wonder if we shouldn't start moving this to a 6 month cycle instead of two. Here are some high level observations based on the previous two releases: 1. It doesn't seem like we are on a true 4 month time based release schedule. Both 4.0 and 4.1 were delayed more than several weeks past the original proposed GA date. 4.0 was released 11/6 and let's assume that 4.1 will ship within a week or two. That's almost a 6 month release cycle. Every release incurs a fixed cost of release notes, upgrade testing, etc. that I suspect at least eats a month worth of time depending on people's schedule. That's 3 months out of the year rather than two if we can get a 6 months cycle. We can use that extra month for other purposes if need be. I suppose if we want to continue to release past the proposed hard GA date, then I guess it doesn't matter if it's 4 or 6 months. It's basically a release when the release mgmt. team feels it's right to release based on current bugs, etc. 2. As more and more features/development go in, it just means more destabilization of the code. 4.0 was delayed and the majority of that work was licensing files. 4.1 got just a bit more complicated with new feature development and the delay is now much longer. Not all features are created equal in terms of testing. Some may require more time to develop but may not impact the entire system like for example, adding a new hypervisor. However, work like refactoring vm sync or other more internal code could affect the entire stack and require more QA time. We need extra time for new code to settle in. 3. ACS is still dependent largely on manual QA. Let's face it, our automated testing/unit testing isn't mature enough quite yet and we cannot always expect manual QA to be there and on ACS schedule. CloudStack releases have some type of quality expectations as well as support for upgrades. Upgrades and migration scripts aren't that easily automatable. Chip and others have been very diligent on ensuring that code check in has the appropriate tests but it's not there yet. 4. ACS development is based on volunteer work and many of us have a $dayjob and may not be able to assist with fixing bugs in ACS schedule. Having only a couple of months to fix bugs and expect others to follow our ACS schedule seems a bit rushed. Wearing my Citrix hat now, I can tell you that 2 months of QA and bug fixing is not enough to release quality GA release. And that is with me breathing down the necks of many of the engineers to get them fixed on time. ACS does not have this type of culture and nor should it. Given that, we should be a bit more flexible in terms of allowing people eventually to act on issues. Will
