Ciaran McCreesh wrote:
Most packages that have tests have working tests. For those that don't, the tests have to be restricted. All this proposal does is ensures that that happens in a progressive, incremental and safe way.
I agree with this point - failing tests are more the exception than the rule.
Looking at my system the only packages I'm skipping tests for are: openldap|parted|orbit|samba|kpilot|nautilus|libksieve|karm|libbonoboui|gnome-vfs|pkgconfig|pam|coreutils|pan|mono|glibc|gettext|curl Some of those might be fixed now.
If packages are failing tests, either it's a legitimate reason, in which case it needs to be fixed, or it's not, in which case it needs to be restricted. The problem is, currently there's no way for users to know which is which. With an EAPI mandated src_test, users will know that any failure that gets to them is legitimate.
Hence my having the list posted above (which is just the ones I use that I've found problems with).
I also would like to say that the "slow-test" compromise sounds like a good idea.
A fast-running automated test routine is a good sanity check to show that nothing went wrong during the build. Maybe the user has some odd version of a dependency that no developer checked with the new package. Arch testers can't test every combination of dependencies, configurations, use-flags, etc.
I would think that this might even cut down on user-reported issues. Better to find out that a package has a problem BEFORE it is actually installed.
If a user is going to spend 10 minutes building a bunch of packages spending another 30-60 seconds on some basic tests doesn't sound unreasonable. We could also make it easy for users to disable testing entirely if they want to live dangerously.
