On Tue, Dec 7, 2010 at 2:41 AM, Robert Bradshaw <rober...@math.washington.edu> wrote: > On Mon, Dec 6, 2010 at 8:43 PM, Niles <nil...@gmail.com> wrote: >> >> >> On Dec 6, 10:53 pm, Jason Grout <jason-s...@creativetrax.com> wrote: >>> Robert (and whoever else is working on the buildbot [1]), >>> >>> First of all, THANK YOU!!! This is amazing! I love how it is hooked up >>> to show the status on trac. >> >> Indeed -- THANKS ! ! ! > > Your welcome. > >>> Could we add building documentation to the things that it checks for >>> each ticket? > > Done. > >> I have a couple of other feature requests/suggestions, which of course >> you're welcome to ignore since this is already great work! But if >> you're interested, here are my thoughts: > > One of the next steps is to get this cleaned up enough to get into > Sage, so people can start improving it and running it on their own > machines. The way it's set up is that there's a single server that can > handle any number of build slaves. > >> 1. When I visit the buildbot page for a particular ticket, I think it >> would be useful to show, in addition to the build/test history, the >> list of patches which buildbot plans to apply on its next run -- this >> would help people verify that the correct attachments/dependencies are >> computed by buildbot without having to just wait for the next run. > > The patches it intends to apply are exactly those listed at the top of > the page. > >> 2. It might be nice to know when buildbot plans to test the ticket >> next -- even a rough estimate would be helpful so that reviewers can >> determine if it would be more efficient for them to build and run >> tests themselves or wait for the buildbot > > That's an interesting idea, but it's not easy to expose that > information. The way it works is that a build slave requests a list of > tickets, rates them according to its own internal settings (which may > be different for each bot), and then starts on a ticket (notifying the > server to avoid overlap). I suppose as well as a pending field list, > one could add something that collects different bots ratings at > different points in time and their respective speeds and does an > estimate. > > Ideally, as well as the "main" buildbot that just churns through > tickets, this would be incentive for people to run their own private > buildbots according to their interests (it can be configured to give > priority to components, authors, etc). I also intend it to be very > easy to run a bot on a list of tickets, which isn't as good as having > the results up there right away, but better than doing things manually > (and the results are shared). Of couse we can't all be pounding away > on sage.math with -tp 12... > >> 3. It seems that there are 2 or 3 tests that consistently fail, and >> which most people ignore for the purposes of reviewing (e.g. the >> startup.py test for startup time on sage.math.washington.edu). This >> means that probably no ticket will ever have "TestsPassed" status, and >> the "TestsFailed" status is less useful because it doesn't distinguish >> between failures which indicate a ticket needs more work and failures >> which should be ignored (for the purposes of reviewing, that is). I >> don't really want to reignite the debate about whether these tests >> should be modified or not, but perhaps buildbot could have a separate >> status when the only doctest failures are from among a certain set of >> files -- this would give more useful information to reviewers. > > Yes. Though it's potentially dangerous to filter on files rather than > individual doctests. The first step would be to establish a baseline > for each server, but beyond that I probably won't have time to > implement much in the near term.
And on this note, I feel that a test that's expected to fail is a bad test--we really need to get to the bottom of this sage0 failure and improve startuptime, hopefully this'll provide some more motivation :). - Robert -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
