Not quite up to the ideal standard of GOP proposals, but there's a lot of interest and this should be enough to see what way the wind is blowing.
html-formatted version: http://lilypond.org/~graham/gop/gop_3.html *** Summary Let’s drop the “any unintended change” thing, and go totally with the regression tests. Tests pass? We can make a stable release. Also, let’s have an official roadmap. *** Motivation There seems to be widespread frustration with the current system. At the moment, any “unintended change” blocks a release (plus a few extra conditions), so we’re at the mercy of all sorts of behaviour that isn’t covered by the regtests. This makes it hard to plan ahead for everybody: developers wanting to work on large features or refactoring, users, linux distribution packagers, etc. *** Details: Critical issues A type-Critical issue will block a stable release, but the definition is: - a reproducible failure to build either make or make doc, from an empty build tree, in a first run, if configure does not report any errors. - anything which stops contributors from helping out (e.g. lily-git.tcl not working, source tree(s) not being available, LilyDev being unable to compile git master, inaccurate instructions in the Contributor’s Guide 2 Quick start). To limit this scope of this point, we will assume that the contributor is using the latest LilyDev and has read the relevant part(s) of the Contributor’s Guide. Problems in other chapters of the CG are not sufficient to qualify as Type-Critical. - any regression test which fails to compile or shows incorrect output. The only change is to the third point, namely the “regression test failure” as opposed to “any unintentional change”. *** Details: Regtests The current regtests don’t cover enough – that’s why we keep on finding new regression-Critical issues. I think it’s worth expanding the regtests and splitting them into multiple categories. These names don’t (deliberately) match any specific testing methodology. If they do, then it’s probably a mistake and we should rename these. Crash: we don’t care about the output of these; we just want to make sure that lilypond doesn’t crash with this input. Tiny: these files would test individual features, such as printing accidentals or slurs, with a minimum of shared features. Integration: these are constructed examples which combine multiple features together. Pieces: musically-interesting fragments of music, such as a few systems from a Bach sonata or Debussy piano work. Syntax: short fragments of music for which the .ly files are “frozen” – we never run convert-ly on these files until LilyPond 4.0. (see below, “roadmap”) I figure that we’ll double the total number of regtests. There’s probably some old ones that can be eliminated (or combined with newer ones), but we’ll be adding a lot more. *** Programming regtests To avoid slowing down programming to a crawl, I figure that we’ll identify some subset of these regtests and have a separate make regtests-quick command which only evaluates that subset. As a rule of thumb, I’d say that the regtests-quick target should take as long as a make from scratch. I’m sympathetic to developers with limited computing resources, but I think it’s reasonable to ask everybody submitting programming patches to “double” the time it takes to test their patch (since obviously everybody would run make before submitting anything). The patchy test-patches will still run the full regtest checks. *** When breakage occurs There will of course be functionality which breaks. When that happens, we file a normal bug. A new regtest can only be added for that bug when it is fixed – we won’t add the regtest first, then try to fix it. In other words, git master should always pass all regtests. If it doesn’t, then reverting should be the first option. *** Roadmap With this change, we would no longer be committed to the same kind of stability that we were before. As such, I think it’s worth bumping the version up to 3.0. The 3.x series will consist of a series of random breakage from functionality not covered under the existing regtests and from manual .ly changes required by GLISS. This is intentional – or rather, we don’t intend to break stuff, but the policy accepts that this will happen. Somebody may offer to maintain the 2.x series to cater to users who want additional stability. Over the next 3 months or so, we’ll discuss a number of syntax changes in GLISS. Then discussion will cease until all the changes have been implemented. We’ll then have release 3.2, which will almost certainly require manual changes to all .ly files. We’ll then have another few months of GLISS discussions, then a pause for implementions, then 3.4. Repeat as necessary. LilyPond 4.0 will mark the ending of GLISS, and by that point we should have much improved regtest coverage. We can’t really plan too much for this, since it’s likely two years away. - Graham _______________________________________________ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
