On Monday 07 July 2008 08:50, Cory K. wrote: > Scott Kitterman wrote: > > I think the only path to better tested releases is higher quality test > > releases. > > Sad fact is people don't install the alpha/betas at the same level as > final. So more actual testing/feedback was done after release. I can > tell you for sure this was the case for Ubuntu Studio. > > For me, it's all about how do we get people to use the later alphas or > betas so final is better? Age old issue it seems. > Agreed. The problem is that the easy solution of moving the problem down a step (fix post-release and .1 is good) is a never ending slippery slope.
Personally, I think it means we need to be doing a much better job of testing and bug fixing as developers. There is one small upstream project that I'm the sole developer for. The first three releases I did were well received and I didn't have any trouble with regressions/failures. Then with the fourth release I started having some significant bugs. Five and six were problematic too. Finally, for the seventh (and current) release, I took the time to write a comprehensive test suite that allowed me to automatically exercise virtually all the code paths. This one I've had no reports of regressions or problems (my test suite did find a major regression I'd missed in my other testing, but it did it before release). The moral of the story is that my project had increased in complexity beyond what my QA approach could sustain and I had to re-engineer QA, even at the cost of some features. Once I had a QA approach that matched the complexity of the project, then release quality went up again. I suspect that we may be in a similar position with Ubuntu. We need to radically rethink testing and how test results get back into fixes. I believe that Ubuntu has gotten more complex and we need to match our test/fix methodology to match. I would like to hear ideas on the subject. There was some discussion at UDS about developing the ability to trivially clone a host machine into a VM so that users could easily test their setups. I can see this being very useful. I am not currently willing to run Intrepid on any of my servers, but if we had a capability like this, I would be willing to take my test server, clone a VM, upgrade to Intrepid, and run with it. That would potentially expose more problems earlier. I'm not certain, however, that we need more bugs. We need better bugs and we need more fixes. Suggestions? Scott K -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
