On Wed, Oct 20, 2010 at 10:24 PM, William Stein <wst...@gmail.com> wrote: > On Wed, Oct 20, 2010 at 9:33 PM, Robert Bradshaw > <rober...@math.washington.edu> wrote: >> On Wed, Oct 20, 2010 at 5:10 AM, Johan S. R. Nielsen >> <j.s.r.niel...@mat.dtu.dk> wrote: >>> I think that Burcin's suggestion is excellent. Development of Sage >>> should definitely move towards more structure, documentation, testing >>> and other software engineering practices, but as for any Open Source- >>> project, these things should come naturally as the project grows and >>> matures; as has already happened with Sage a lot, it seems (for a >>> relative new-comer like me). To require too much too soon would kill >>> the joy of working on the project and thus kill the project itself. >> >> +1 I have definitely seen that the level of bureaucracy has going up, >> especially in the last year or two, has turned off a lot of potential >> and even former developers. > > Another +1. The level of bureaucracy has gone up so much in the last > year that it has very seriously turned off me. > >> The focus on software engineering and >> testing can certainly be good for quality (though that's not an >> immediate or certain implication), but the problem is that too much >> emphasis on it has a significant chilling effect on contributions. The >> lag time between hacking out some code and getting it in is way too >> high these days discourages contribution and sucks up a lot of >> development time and energy with endless rebases and waiting. > > +1 > > It kills a large portion of potential contributions of code for > advanced research, perhaps 80% or more. I've talked to sooooo many > people about this in the last few months... > >> And >> though we all want to produce bug-free code, holding that up as the >> primary objetive (as opposed to producing useful code) I think >> dissuades people from submitting or refereeing code. > > Also, anybody who has significant experience with software engineering > knows that producing bug-free code is a recipe for producing almost no > code. > > I was talking to somebody today who works on Microsoft Windows > (actually implementing the OS there), and who has also written a lot > of code for Sage (at a "research level" -- advanced number theory > stuff). He said internally at Microsoft code gets into the system, > and out getting used by a lot of people (internally) much more quickly > than with Sage. Instead of the very "all or nothing" model that we > tend to have, they have many levels of review that code goes through. > Sage would benefit from something similar. That's basically what > http://purple.sagemath.org/ is about: a way to get code out there and > distributed, used, etc., but without all the bureaucracy. As an > example, I'll sit down this coming Tuesday with Sal Baig, and get his > and Chris Hall's library for computing with elliptic curves over > function fields into PSAGE, and have it be in the next release. > That's code that isn't stable yet and is mainly for research. For a > year they have been trying to get it into Sage, but it just isn't > happening, since they care and know much more about *using* the code > for research purposes, than about how to make a proper > makefile/autoconf setup, so it builds fine on Solaris and OS X 10.4. > > I think PSAGE will show that the increasingly bureaucratic and > plodding way in which code gets into Sage isn't necessarily bad, in > the same sense that Debian can be very plodding and bureaucratic, but > it still provides a good foundation for many other much more svelte > and useful Linux distributions. > >> I'm not sure I have a solution, but one thing that keeps coming to >> mind is that Sage is trying to span several audiences with different >> goals and criteria, and perhaps the various audiences would be best >> met by a stable vs. unstable model like Debian. Purple sage is an >> extreem move in this direction (more like Debian experimental)--I can >> certainly see the attraction and value, but I just hope it doesn't >> become an incompatible fork. > > A difference from debian experimental is that PSAGE starts by removing > over 20 standard packages from Sage. In fact, so far, that is > essentially *all* that PSAGE is. Also, my intention is that most > Python code in PSAGE go into a different Python module than the "sage" > one.
Thus psage will be a superset of a subset of sage. Do you envision a migration path of code from psage to sage? (Perhaps not instigated or executed by the original authors of the code of course.) Would it be easy to install the "missing pieces" into a psage setup, or, conversely, install the new parts of psage on top of Sage? That being said, the sage-combinat model seems like it would be a huge amount of work to manage, but is nice for the end user. (Doesn't solve the messiness of ugly build problems with arcane spkgs...) >> First, have the initial status of tickets be some pre-new stage. >> (Something like "unclassified".) This woud make it so you don't have >> to be an expert to classify a bug. Volunteers could go and look at all >> unclassified tickets and file them appropriately (severity, defect vs. >> enhancement, component, duplicate, etc.) Of course, there could be a >> rotating bug-wrangler, but if it was easy enough for a "veteran" to >> hop on and categorize a bunch of them in one sitting this might not >> even be necessary. > > This is perhaps already provided by: > > http://spreadsheets.google.com/viewform?key=pCwvGVwSMxTzT6E2xNdo5fA > > Harald Schilly could probably set things up so more people can wrangle the > bugs > that come in through that page. I think that's rather orthogonal to the number of poorly filed tickets on trac. >> Second, rather than have the default milestone be the next release, >> have some default future milestone. > > There's no default milestone. See for yourself: > > http://trac.sagemath.org/sage_trac/newticket > > It being the next release is merely a social convention... That said, > of course you mean "instead of whatever is done now..." Yes, that's what I meant. There should be a good (non-blank) default, as it currently does scream "change me." >>Only tickets that are actively >> being worked on get put on the current release. This would make it >> much easier to see what's being worked on (or at least cared about) >> and what to expect in the next release. Sufficiently important items >> (blockers) would also get added here. > > Couldn't something like that be accomplished with a trac report > showing the 50 "most active" tickets? Sort of like how > http://ask.sagemath.org/questions/ by default shows the 50 most > active questions, and http://mathoverflow.net/ shows the top 50 or so > most active questions? I think activity is not always an indication that something will go in soon (though it would still make for an interesting report). Also, release managers should feel free to bump tickets out of a given release whenever they not comfortable with them (whether that be due to lack of expertise or time or wanting to roll things to an end or whatever. As a minor point, I also like the line-bar plot by component releases have. >> Thirdly, I think the list of components could be cleaned up. > > For the last year people kept asking to add new trac components (I > know, because I had to add them). There used to only be a handful. > > Just out of curiosity, is trac still the best system to use for > managing what we're managing? Probably not, but I don't think our issues could be sovled by switching systems alone (though it's probably worth doing). I do have to say that mondrian (a.k.a. code review) is really nice to work with, and I have to say that code reviews there are a much smoother process than what we have with Sage. One key point is that it's far, far easier to just fix and repost/refresh patch. > You work at Google and they have their > own free http://code.google.com/ thing, which provides similar > capabilities to trac, but with integrated code review, the possibility > to delete comments (w00t!), etc., native support for Mercurial (which > trac barely supports, preferring SVN), easy forking, etc. I have so > far little experience with code.google.com, but I'm sort of curious > how good it is. I set it up for PSAGE here > http://code.google.com/p/purplesage/ since I didn't want to have to > manage a whole trac, hg repo, etc., yet again. > >> All of these would help us get a better picture of the overall state >> of Sage. Finally, we need more automation. Refereeing code shouldn't >> have to involve downloading and applying patches and running all >> tests--that should all be done automatically (with failing tickets >> bounced right away, or at least in a 24-48 hour window). We should >> have a notebook server with sessions of Sage with various tickets >> already applied for quick refereeing (or a CLI interface on, say, >> boxen for those who prefer that--"telnet/ssh >> 10921.sage.math.washington.edu" would be cool, opening you immediately >> into a jailed sage session). I'll plug >> http://trac.sagemath.org/sage_trac/ticket/9967 which is a requirement >> for this and I just recently started to write a per-ticket build bot. >> This would help close the gap and lag between writing code and getting >> it into Sage, and I think that having such a small gap was one of the >> key ingredients in letting Sage explode like it did. > > +1 You hit the nail on the head. The small gap absolutely *was* the > reason. I missed so many key things in (social and other) life > during the first 1.5 years of Sage in order to make the next releases, > because I knew how absolutely critical it was putting together new > releases in order to draw in developers and keep the momentum going. > Making a release 1 day earlier, literally made a huge difference > during that time. Many developers who got involved then, because of > the fast cycle, have since stopped working on Sage. Now the release > cycle is about 2 months between releases. You can see here (look at > the dates) what it used to be: http://sagemath.org/src-old/ > > Again, I'm not necessarily claiming Sage itself needs to move that > quickly again. This is very difficult technically due to the larger > number of platforms supported, the larger test suite, codebase, etc. > But something does need to change, or some of the truly brilliant > computational mathematics researchers (like Mark Watkins, say) will be > unlikely to be drawn in again. For me, PSAGE will accomplish this > goal, while allowing me to continue to benefit from the incredibly > valuable high quality work that so many people are doing on making > Sage a solid, well tested, cross-platform system. I agree, something needs to change, and it makes more sense to create an agile offshoot. I suppose my sweet spot would be where Sage was 2-3 years ago: reviews were in place but usually quite quick, doctests were good but 100% was not required, less worrying about the long tail of operating systems. etc. (I'm probably suffering from golden-age nostalgic blindness a bit here...) But that may not be what's needed, especially to jumpstart things again. I just don't want to see psage becoming a divergent fork of what Sage was in late 2010, or an enormous amount of effort required to keep the two projects on a track where they can continue to benefit from each other. - Robert -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
