tl;dr old curmudgeon flaming on about the dead past, not "getting it"
about Sage.
Robert Bradshaw wrote:
In terms of the general rant, there are two points I'd like to make.
The first is that there's a distinction between the Sage library
itself and the many other spkgs we ship. By far the majority of your
complaints have been about various arcane spgks. Sage is a
distribution, and we do try to keep quality up, but it's important to
note that much of this software is not as directly under our control,
and just because something isn't as good as it could be from a
software engineering perspective doesn't mean that it won't be
extremely useful to many people. Even if it has bugs. We try to place
the bar high for getting an spkg in but blaming the Sage community for
poor coding practices in external code is a bit unfair. I hold the
Sage library itself to a much higher standard.
The point that "software is not as directly under our control" is not
really valid.
This is a *design* decision of Sage, not a necessary evil. Sage
specifically chose
to build on dozens of other pieces of software. Once you make spkg
functionality
part of Sage functionality, you own it.
The statement that Sage tries "to place the bar high for getting an spkg
in" isn't
actually much of a claim. I've watched the way spkgs get voted onto the
island
and it usually involves a +1 by less than half a dozen people. Would you
really
consider this to be placing "the bar high"? I'd consider developing a
test suite,
or an API function-by-function code review, or a line-by-line code review to
be placing the bar high. At the moment I see Sage writing test cases for
python
code but I don't see the same test cases being pushed into the spkgs.
Even where
external test cases are available (e.g. the computer algebra test suites
for Schaums
and Kamke) I don't see them being run.
From a software engineering perspective there are some things that *are*
directly under Sage control such as the pexpect interfaces. How carefully
are these designed? Just yesterday I saw comments about the gensym
(question-mark variables) connections to Maxima not being handled. This
syntax is not a new Maxima feature so a pexpect interface design could have
taken this into account but it did not. Each pexpect interface should be
designed
to be automatically constructed from the BNF of the underlying spkg. This
would eliminate the mismatch by design and be good software engineering.
The conclusion that "blaming the Sage community for poor coding practices
in external code" as being "a bit unfair" is not valid. While it is
grossly unfair to
assume that spkgs are of poor quality, if your *design* calls for using
materials
of "unknown quality" it seems that a very large portion of your effort
*must*
involve quality reviews of spkgs. End users just see Sage.
Still to come will be the "code rot" issue. Open source packages tend to
have a
very small number of active contributors. Projects tend to stop when
those people
drift away. Once a package is no longer maintained it stops working due to a
lot of factors such as incompatible standards like python 3.0, operating
system changes
like include files, architecture changes like parallel builds, loss of
primary
development platforms like the implosion of open solaris, etc. Recent
examples of this
in Sage might be the Atlas 64bit issue (architecture), the Sympow issue
(author
loss), the loss of pointful effort due to the death of open solaris
(platform death),
the python GIL issue on multicore (software architecture), the rise of
python 3.x
(software standards change), etc.
Now that the wave of new spkg adoption has slowed I expect to see a growing
need for maintaining "upstream" code. By *design*, their problems are
now your
problems. Who will debug a problem that exists in 500,000 lines of
upstream code?
Who will understand the algorithms (e.g. sympow) written by experts, some of
whom are unique in the world, and debug them?
Writing new code is always fun. Maintaining old code you didn't write is
painful.
But from an end-user perspective "it is all Sage" so all bugs are "Sage
bugs".
That may seem unfair but the end-user won't know or care.
The belief that Sage will gradually rewrite the code pile it has (5
million lines?) into
higher quality seems odd. For comparison, Axiom is about 1 million
things-of-code
(lisp doesn't have "lines"). It took over 20 years and over 40 million
dollars of funding.
Scaling linearly, Sage would take 100 years and 200 million dollars to
be rewritten
into "all python". Frankly, I think the spkgs are going to be around for
a very long time.
The second point is that much of the older, crufty code in Sage was
put in at a time when standards were much lower, or even before there
was a referee process at all.
When Axiom was written we were using Liskov's ideas directly from the
primary papers.
I believe that we were the first system to dispatch not only on the type
of the arguments
but also on the type of the return (something that is still not common).
But Axiom was
developed as research software, not with the intention of being brought
to market as a
product (free or commercial). Sage is being developed with this intention.
Our choice of "standards" was to build on abstract algebra. There were a
great many
debates about the right way to do things and we always went back to the
touchstone of
what abstract algebra implied. At the time (40 years ago) there were no
existing examples
of computational mathematics for many of the ideas so we had to invent
them. Axiom
set the standards (e.g. integration) and they were quite high (Axiom
still has the most
complete implementation). Sage has existing examples to guide it.
So at the time Sage was being developed there *were* standards in place.
You seem
to feel that Sage was started "pre-standard" (2005?) and "pre-referee"
(ISSAC?).
I think this was necessary for the
time--Sage would have gotten off the ground if it couldn't have been
useful so quickly. This includes in particular many of the spkgs that
have been grandfathered in and wouldn't make the cut now, but it takes
time to remove/replace/clean them up. Of course there's room for
improvement, but do you think the current review process is
insufficient and lots of new bad code is being written and included?
If so, what should we do better?
I *do* feel that the current review process in Sage is insufficient (see
my earlier diatriabe).
I see reviews of bug fixes but I don't see reviews of spkgs. We are now
over 50 years
into the development of computational mathematics and Sage has the goal
of competing
with systems developed in the 1970/1980s, over 30 years ago. This would
be a great
thing if Sage were to deeply document the algorithms, develop the
standards, and/or
prove the code correct but I don't see anyone advocating any of these. I
don't see anyone
advocating alternative ideas that would "raise the bar" in computational
mathematics.
Even in the area of education I don't see anyone hammering on the NSF to
fund more
efforts in computational mathematics. I don't see pushback to NIST to
standardize the
algorithms. Obama wants to bring science back to life and encourage
research. As the
largest group of academics I would wish that you would petition the
funding sources.
Even if all of the funds went to Sage I'd still feel that this was
worthwhile.
In short, I don't see *change*.
Tim Daly
(curmudgeon and rjf wannabe)
--
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org
