Robert Bradshaw <rober...@math.washington.edu> writes: > On Fri, Mar 2, 2012 at 5:35 AM, Keshav Kini <keshav.k...@gmail.com> wrote: >> I would actually like the patchbot to NOT do continuous builds and >> tests, once we move to a push/pull system. There should be a big button >> on each ticket which says "Test Me!", which will cause the patchbot to >> fetch whatever branch is specified on the ticket (in some "branch" >> field) and put the pair "(ticket, commit-id)" into a queue. (The >> patchbot wouldn't actually test commit-id, but would attempt to >> temporarily merge commit-id into master, and then test the resulting >> commit or return a failure to automatically merge / report that the >> branch needed merging / rebasing.) > > I think that each patchbot could, according to its configuration, > periodically pull a newer version of master (and run tests against > that to isolate faults before trying to apply tickets). This is > essentially what we do now, but "master" is whatever sage version (or > alpha) you're running and doesn't change. (This also bleeds into the > discussion about being able to upgrade sage and all its dependencies > by syncing a single repository...) > >> One reason for this: once we switch to a push/pull system, commits will >> hopefully be a lot more frequent, and I question whether we would even >> want the patchbot to test each and every commit over the whole doctest >> suite. > > Setting a ticket to "needs review" is basically this button, I don't > think we need a separate one.
The difference is that tickets do not automatically get set back to "doesn't need review" when you push new commits, nor does the patchbot get notified, necessarily, when you push new commits. This should fundamentally be a "request action" type of thing, not "set status" type of thing, IMO. > We wouldn't test "every commit," it's > the tips of what gets pushed (or, possibly, when the master progresses > forward enough). I don't want pushing my commits to lead to side-effects. People should be encouraged to push their commits somewhere public as soon as possible so people can see what they are doing and collaborate. > As for wanting to test aboundently, there are two > possible concerns: > > (1) the utility of running tests, and > (2) the cost of running tests. > > In terms of utility, even if a ticket is not "done" it can be useful > to see what its status is, especially if it affects distant parts of > the library (e.g. breaks monsky-washnitzer...) or for collaboration > (or if a lot of time has passed) do see what is broken or if it no > longer applies. True. > As for cost, one thing that was far from obvious before actually > writing the patchbot is that testing everything is surprisingly cheap. > I mean so cheap that a single computer can keep up with everything and > still sit idle most of the time. If this made things an order of > magnitude more expensive than we could still keep up by using less > than a dozen cores (on sage.math, or a handful of developers willing > to run niced processes on their machine). We have almost the opposite > problem: I'm willing to donate CPU cycles and there's nothing for my > machine to do right now. Wow, I had no idea. That's certainly good to hear. > The other thing to keep in mind is that the patchbot is structured so > the client decides what tickets to accept (and what the relative > priority of each should be). This way high priority tasks (e.g. > needs-review tickets with no background) never have to take a back > seat to lower-priority ones; the lowest priority ones would only > happen when there's "nothing better" to do. Also, clients can decide > what threshold they're willing to contribute at. (This would be > especially useful for an expensive but unique system that might want > to only check positively reviewed tickets once.) In that case maybe we could set manually requested tests at a higher priority and automatic tests at a lower priority. The automatic tests would be silent if successful and result in a warning comment to the ticket if something went wrong, whereas the manually requested tests could post to the ticket a link to a full log of the doctest. I would want to avoid automatic doctests marking anything as "failing", though. The current red/green/yellow status should be based on the last manually-requested doctest, not the last doctest. Perhaps green and red for "last manually requested doctest passed" and "last manually requested doctest failed", respectively, with an overlaid question mark if the branch set on the ticket is not currently on the commit which was last manually tested. We could make the question mark green or red depending on the status of the latest autotest... I'm almost literally painting a bikeshed here, haha. >> Of course, it would be fine for the patchbot to at least notice >> that its last test was out of date and mention this on the trac ticket >> somewhere - maybe we could have a little "patchbot info area" <div> >> instead of the current patchbot icon. > > There's only so much info you can pack there (or elsewhere on the > page). We could stick a little bit more, but this involves modifying > trac and has diminishing utility over just putting detailed status on > a separate page that's only a click away. Right, that's fine too, of course. -Keshav ---- Join us in #sagemath on irc.freenode.net ! -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
