On Mon, Jun 14, 2010 at 5:17 AM, Dr. David Kirkby
<david.kir...@onetel.net> wrote:
> On 06/14/10 12:18 PM, Tim Joseph Dumol wrote:
>>
>> On Mon, Jun 14, 2010 at 7:09 PM, Dr. David Kirkby
>> <david.kir...@onetel.net> wrote:
>>>
>>> On 06/14/10 11:41 AM, Tim Joseph Dumol wrote:
>>>>
>>>> Doctests are used to prevent regressions and (unwanted) backward
>>>> incompatibilities. Since the code used in these modules are not ever
>>>> going to be modified, it does not seem necessary to provide doctests,
>>>> IMHO.
>>>
>>> Personally I'd beg to differ. A change in gcc's behavior could easily
>>> result
>>> in the code acting differently, as could any number of other system
>>> changes.
>>>
>>> Here are a few tickets for issues that result of just changing compiler
>>> versions.
>>>
>>> * segfault in Sage-4.4 built using GCC-4.5.0
>>> http://trac.sagemath.org/sage_trac/ticket/8788
>>>
>>> * frobby optional spkg doesn't build with newer GCC's
>>> http://trac.sagemath.org/sage_trac/ticket/8783
>>>
>>> * GCC-4.5.0 breaks GAP -- the workspace is broken, hence gap('2+2')
>>> fails.
>>> http://trac.sagemath.org/sage_trac/ticket/8773
>>>
>>> * http://trac.sagemath.org/sage_trac/ticket/8767
>>> http://trac.sagemath.org/sage_trac/ticket/8767
>>>
>>> As far as I can see, all those bugs were a result of changing just
>>> compiler
>>> versions.
>>>
>>> Add to the mix the possibility of different versions of cython behaving
>>> differently, and it seems a bad idea to me.
>>>
>>> It's certainly not unknown for a doc test to fail on one machine but pass
>>> on
>>> another. So having the same code never guarantees you get the same
>>> result.
>>>
>>> Over the years, I've come across a lot of code which runs ok on fast
>>> computers, but not on slow ones, or visa versa. One case I recall was
>>> someone being rather stupid and seeing the random number generator from
>>> the
>>> time of day multiple times in a loop. On a slow computer, the seed was
>>> effectively random each time so they got a different pseudo random
>>> number.
>>> On a fast computer, the code executed in less than a second, and so the
>>> RNG
>>> was seeded twice with hte same value.
>>>
>>> Mathematica on Solaris had a bug when Solaris 10 was updated only on slow
>>> computers.
>>>
>>> http://www.g8wrb.org/mathematica/
>>>
>>> So I've known all these to cause bugs, while the source code remains
>>> unchanged.
>>>
>>> * Changes in compiler version
>>> * Changes in the speed of the computer
>>> * Upgrade of the operating system.
>>>
>>> As one more final point, there are ports in progress to
>>>
>>> * FreeBSD
>>> * OpenSolaris
>>> * 64-bit on Solaris SPARC
>>>
>>> All of them have the potential to create problems on one platform, not
>>> seen
>>> on another.
>>>
>>> Can you dismiss all the above possibilities? If not, why should the code
>>> be
>>> exempt from testing?
>>>
>>> Dave
>>>
>>
>> As for Cython and gcc, the Sage notebook uses pure Python. I do
>> acknowledge that there's a minuscule chance that a Python update could
>> change runtime behaviour.
>
> But what is used to build python? - gcc of course! So we have *at least* the
> following possibilities which could result in a problem.
>
> * gcc update
> * python update
> * someone patching python (it is already at patch level 8 or so in Sage)
> * operating system update
> * port to another platform (Cygwin, OpenSolaris and FreeBSD are all being
> worked on.)
> * someone's computer may be mis-configured.
>
> Less likely, but still not impossibe, would be the speed of someone's
> computer (BSD.py was such an example), or any of numerous other things I can
> think of.
>
>> It is worthwhile to note that the code under sage/server/* is only
>> used to be able to load old pickles of Sage notebooks, and the only
>> reasonable way I could think a Python update could mess this up is by
>> a change in pickle format (which is guaranteed against in Python
>> documentation). The code is not used for any other purpose aside from
>> that.
>
> Maybe, but it seems a poor idea to remove it to me the fact the code is
> still used - even if only rarely used.
>
> What do we gain this from removing this code from doctesting.
>
> * Faster doctesting.
> * Better looking statistics.
>
> I know what I'd rather have.
>
> Is there *any* other motivation for removing this from the testing, apart
> from increasing the percentage of doctest coverage? If not, it boils down to
> sacrificing quality for better looking statistics.
I very, very strongly support Tim's position that we should not
include the old Sage notebook pickle code in the coverage score.
1) His arguments, namely that the code isn't actually used. It is
only the classes in the files that are used to load old notebook
pickles (the code isn't even run). At some point being able to load
these will be deprecated too.
2) Even if we were using that code, it would be foolish to include
it in the doctest coverage code. The standard "doctest" approach to
code testing makes a lot of sense for the math part of the Sage
library, which is meant to be used interactively. It is very awkward
to test the notebook server code, which is not used interactively at
the command line. Instead, for the notebook server, it is much better
to have (a) unit tests, and (b) use Selineum to do functional testing
from a browser.
3) Including the old notebook server code in the statistic
encourages people to write doctests for that code. If anybody did
this, it would be a complete and utter waste of valuable time.
That said, David does have a very good point, that via some weird
sequence of events the code in sage/server could get broken. Since
that code is supposed to serve exactly one purpose now, the one and
only test that we *should* have of that code is that it can be used to
unpickle an old Sage notebook directory. I've opened a ticket for
this:
http://trac.sagemath.org/sage_trac/ticket/9237
-- william
--
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org
