First of all, what sets us apart from most open source projects is that the time required for a CI (buildbot) run is about a day. And we produce more more tickets than one per day. So unless we massively throw hardware at this, we have to collect a couple of tickets for each buildbot run and then backtrack if there is a failure.
The biggest flaw of the current process is IMHO that not enough things are automatized; Most build failures are actually trivial (like documentation doesn't build) and should be caught without human interaction before doing a full buildbot run. The problem is that our tests are too flaky to be 100% reliable. For example #23391 often causes at least one buildbot to crap out. And patchbots often produce spurious errors, which leads to people ignoring their output. In part this is because we don't have a lot of incentive to hunt down rare failures, most people just want their ticket merged and don't run tests 100x. Another sign of things not being automated is that tarball updates require human (i.e. my) intervention to upload them. And as long as they aren't uploaded they can't be tested. To address these issues, I/we should: 1) address the tarball issue, either add a download url to, say, checksums.ini and code to auto-download from that url if it can't be found on the mirrors. Or set up a web service where tarball updates must be submitted to be accessible by build/patchbots (any volunteers for hardware / hosting?) 2) work towards a release script that automatically merges or sets back to "needs review" based on tests on a fast machine; This will also increase the visibility and pain of random testsuite failures... -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
