On 06/10/10 09:25 PM, Minh Nguyen wrote:
Hi folks,
One of the main goals of the upcoming Sage 5.0 release is to get
doctest coverage of the Sage library up to at least 90%. As of Sage
4.4.4.alpha0, the overall weighted coverage is 82.7%.
Seems we are a long way off.
It seems to me, rather than pick a number like 90%, the areas should be targeted
carefully.
To get a sense
of which modules in the Sage library need work on their coverage
scores, you could use the coverage script as follows:
$ ./sage -coverage /path/to/module.py[x]
Or you could do the following to get the coverage scores of all
modules, including a coverage summary:
$ ./sage -coverageall
You might be interested in knowing which modules have a certain
coverage percentage, in which case you could save the output of
-coverageall to a text file and then grep that file for certain
coverage scores. At this repository [1] is a script to generate
various types of coverage analysis reports. You can also find the
script at [3]. The script currently supports the following reports
* The coverage summary of all modules.
* Modules with 100% coverage.
* Modules with zero coverage.
I don't really understand these docs tests well, but to me at least,
concentrating on modules which have zero coverage would seem best, even if only
one doctest/module is added. My logic being something could be totally broken,
and we would never know about it. At least if there is even one doctest, it
shows it is not totally broken. (Though one could argue being totally broken is
better than being partially broken. At least one finds out in use.)
It was recently discovered that certain modules (matrix, class, mgcv, nnet,
rpart, spatial, and survival) in R are not building on Solaris.
http://trac.sagemath.org/sage_trac/ticket/9201
Had there been even one doctest for each module, this would have been obvious.
There is an interesting information about the SQlite database (incluced in Sage)
is tested. The number of lines of test code is 679 times that of the actual
source code of the project. The number of lines of test code is 45.7 million,
the number of lines of source code in the database is 67000! (Both exclude
comments and blank lines)
http://www.sqlite.org/testing.html
I think their test procedures are a bit over the top, but it certainly brings in
to perspective how some developers feel about testing.
If what is written at
http://reference.wolfram.com/mathematica/tutorial/TestingAndVerification.html
about testing Mathematica is true, then the following statement is intesting
"There is also a special instrumented version of Mathematica which is set up to
perform internal consistency tests. This version of Mathematica runs at a small
fraction of the speed of the real Mathematica, but at every step it checks
internal memory consistency, interruptibility, and so on"
I must admit, reading that Wolfram Research page, the statement that "The
standards of correctness for Mathematica are certainly much higher than for
typical mathematical proofs" is extremely stupid, when they don't define
"typical" and they provide no evidence of it. (It was not me he spotted that,
but it is extremely dumb thing to write)
Of course we can't verify the claims made by Wolfram Reserach, but we can verify
what the SQlite developers say, that the number of lines of test code is 679 x
the amount of actual code in the database.
Even I would refuse to write 679 lines of test code for every line of code I
wrote! But really the specification, implementation and testing should be done
by different people. In practice, that is not going to happen in Sage, though I
would not be surprised if that happens with Mathematica, since it is pretty
standard technique in software engineering.
Dave
--
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org
