Hi Neil, On 2020-01-16 5:04 p.m., Neil Bowers wrote:
This also points out that we could do with some kind of monitoring system, to spots sudden changes in CPAN test results (without a new release). In this case lots of people noticed that things started failing, but earlier warning in the right place would be useful.
I maintain about 100 CPAN dists for my employer (not on this account), and that was starting to get a bit unwieldy, so I came up with a system similar, I think, to the one you are proposing. I parse and store the CPAN Testers log data to check for recent failures (with some simple heuristics in case one tester is having trouble, as sometimes happens). If there are failures on any of the modules, it generates an email report linking to the failed distribution(s).
Since everything is done off of a local database, scaling to hundreds of thousands of distributions is no problem performance-wise. Adjusting the output a bit will of course be necessary, but there is opportunity to, say, look for only "new" failures like you suggest, send emails to the author(s), send either a summary email to modules@, or a "hot topics" report that looks at failures + reverse deps over threshold to really light a fire under modules like TimeDate that need more immediate attention.
Is anyone already working on something similar? If not, would there be interest from the admins in having me adapt what I've done?
- R
