Adam Kennedy wrote:
Likewise, if your module installs all the way from a vanilla
installation and all it dependencies go on cleanly, then I think that's
well and truly worthy of a point.
Something like a clean_install metric. If there are any FAIL entries in
CPAN Testers against the current version of your module, you lose a point.
Those two are not the same. Leaving aside that Kwalitee tests don't run
code, the ability of a vanilla version of the latest production release
of perl to install a module and all of its dependencies with the vanilla
version of CPAN for that release could be an interesting signal of quality.
Knocking off points for fails, however, might be due to things that are
completely idiosyncratic. For example, anyone whose module depended on
a test module that used Test::Builder::Tester when Test::Builder changed
and broke it could get dinged.
Does this really tell us anything about actual quality?
What about if I list a prerequisite version of Perl and someone who
tries it under an older version causes a "FAIL" on CPAN Testers? Does
that tell us anything?
There are so many special cases that I don't think the value derived
from such a metric will be worth the effort put into it.
The vanilla testing is a more interesting idea in its own right. I've
had that on my back burner for a while -- installing a fresh perl and
scripting up something to try installing my distributions to it, then
blowing away the site library directory afterwards. I just haven't
gotten around to it, so I look forward to seeing what you come up with.
David
