Chromatic and I were discussing CPAN testing at a conference yesterday. I made some comments about things that I thought would be useful and he encouraged me to post them here...
Default tests - Like probably a lot of perl module writers, I don't write tests nearly as often as I should, but I have collected a few simple default tests that I can toss into a new module and reuse without modification. For example: http://search.cpan.org/src/BRYCE/Test-Parser-1.00/t/00_initialize.t http://search.cpan.org/src/BRYCE/Test-Parser-1.00/t/01_script_compile.t http://search.cpan.org/src/BRYCE/Test-Parser-1.00/t/zz_dump_config.t It would seem to be useful if CPAN were to have a couple really basic tests like this to run for perl modules that haven't created any tests themselves. Obviously, this only benefits immature modules, but this might be a good way to motivate new authors to put more attention into testing. Beta testing - CPAN tracks intra-module dependencies, and does automated testing, but only on released modules. It would seem useful to allow an author to upload a "beta" version of their module, and then run the tests of a dozen or so modules that depend on it. This way, you would gain confidence that your next release won't break everyone else's code (or be able to explain why, if it must.) Obviously, this testing would mostly just benefit core modules, but you could argue that those are probably the ones most in need of solid QA. This could save worry time for the core module authors, allowing them to focus more of their energy into improvements. Automatic module patching - Wikipedia has a process for creating "bots" to do certain mundane updates across a number of pages. For instance, one bot inserts information from the CIA Factbook for all cities and nations. It would be interesting to have mechanisms like this that browsed through the module list looking for common omissions or errors in modules, and then automatically send the author a patch to fix it. For instance, if the author didn't properly fill out the COPYRIGHT, , or SYNOPSIS, a tool could generate a patch with some suitable guesses. Or, a script could google for published articles about or mentioning the module, and create a patch that lists the top 5 or so in the SEE ALSO section. I imagine this process could become extremely invaluable for doing large scale upgrades to bring modules forward into Perl 6. For instance, if it is decided to switch MakeMaker or other low level modules, then the ability to programmatically generate patches to upgrade modules would probably help push authors to upgrade their modules much faster. Bryce
