On Sun, 2005-09-04 at 20:12 -0500, Daniel Goller wrote: > sounds like you suggest to trick ~arch users into testing "unripe" > ebuilds/bumps/versions by sending it into ~arch to get the testing done while > someone in a chroot would be much better equipped for doing the testing with?
No. You've got to look at it in the context of the environment we work in. The vast majority of our packages do not come with test strategies, comprehensive test scripts, automated test cases that provide a large coverage of the code base, or any other QA tools that a career software QA person would expect / desire. We don't have activities in place to fill this gap - it's probably beyond our very limited resources anyhow. We don't require (and don't cover in the quizzes) devs to have any training or experience of deterministic software testing, or of regression testing. It is not our policy to require devs to write regression tests for every valid bug posted in Bugzilla (or for any bug at all, for that matter). A package maintainer does what testing he can, and there are times when we'd all wish the maintainer had done more :) But there will always be the need for a wider audience to be encouraged to test the package. A good rule of thumb is that a QA budget should be 40% of the development budget. We don't have the manpower to come anywhere near that. We have only a fraction of the resources of Debian or RedHat - there comes a point where we have to make up the difference *somehow*. For better or for worse, historically in Gentoo we've done it by turning to our users. Many *users* look on package.masked packages as being dangerous to install, but are much more willing to run ~arch packages. If you mask a version of a popular package, you'll get a lot of correspondence asking you when you'll unmask it, but you won't get much testing feedback to go with it. Once the package moves to ~arch, the amount of feedback improves substantially. If a package maintainer believes that a package WORKSFORME *after due diligence*, then he's not only entitled to move it to ~arch, but he's got no reason not to. I've been through this first hand over the last 14 months with PHP 5. It's a big package, one that I use all day every working day, and a topic I co-wrote the official certification study guide for. It's fair to say that it's a package I know a lot about, and that others in the community recognise I know well. But, with over 100 separate features to enable and disable, even over 14 months there are large parts of the package that I won't have tested in depth, and there will always be features that I'll never touch. I kept PHP5 masked for those 14 months, and (as Jakub and others can confirm) most of the feedback has been limited to "unmask that puppy" (sometimes put in stronger terms ;-) There were some bugs from users who had found issues, but not many. Rather than unmask the packages before they were read, I changed to another approach. I moved the work out of Portage into an overlay instead. This worked well. It has attracted a bunch of regulars to #gentoo-apache who have spent the last few months finding the bugs that existed, and making sure that they're fixed and stay fixed. It looks likely that we'll get some new devs out of that too :) Overlays are easy for larger pieces of work like PHP, Java, Gnome/Gentopia, but they'll always be small packages where an overlay feel like too much effort. They may not be the answer to everything. If we'd seen through the policy that every package has to belong to a herd, then we could organise overlays by herd - and maybe leave it up to the arch teams to import "stable" packages from the overlays into the Portage tree proper. Best regards, Stu -- Stuart Herbert [EMAIL PROTECTED] Gentoo Developer http://www.gentoo.org/ http://stu.gnqs.org/diary/ GnuGP key id# F9AFC57C available from http://pgp.mit.edu Key fingerprint = 31FB 50D4 1F88 E227 F319 C549 0C2F 80BA F9AF C57C --
signature.asc
Description: This is a digitally signed message part
