On Feb 20, 2010, at 3:29 PM, Dr. David Kirkby wrote:
The thread 'Sage 4.3.3.alpha1 released' which covers loads of stuff.
The particular bit I'm referring to is this test failure failure
reported by
Robert Marik.
----------------------------------------------------
----------------------------------------------------
----------------------------------------------------
sage -t "devel/sage/sage/misc/functional.py"
**********************************************************************
File "/opt/sage-4.3.3.alpha1/devel/sage/sage/misc/functional.py", line
705:
sage: h.n()
Expected:
0.33944794097891573
Got:
0.33944794097891567
**********************************************************************
1 items had failures:
1 of 14 in __main__.example_28
Robert Marik
----------------------------------------------------
----------------------------------------------------
----------------------------------------------------
The test reads like this:
##############################################
Numerical approximation::
sage: h = integral(sin(x)/x^2, (x, 1, pi/2)); h
integrate(sin(x)/x^2, x, 1, 1/2*pi)
sage: h.n()
0.33944794097891573
#################################################
The test certainly does *not* test Sage's ability to correctly
calculate that integral. It tests the documentation gives the same
answers each, but that is not a lot of use of those answers are
incorrect. Nobody appears to have bothered to evaluate if they are
correct or not.
Part of refereeing a patch is making sure that the tests in the
documentation are correct--if people are not doing this then they are
not doing a proper job refereeing what goes in. This means more than
just trying out the relevant command and seeing that it gives the
predicted answer. (Note that here the test is really that .n() works
on an integral, not that the numerical routines used are stable.)
It might verify that Sage gives the same results each time, but
without knowing what the exact result is, this is not a very good
test.
The exact answer can't actually be written down. I wouldn't say that
this answer is incorrect, just not as precise as you might want. Now
you might want every printed digit to be correct (which I think is a
valid concern, and I think n() should have this property) but it's not
a distinction that is currently made in Sage, and it's not so easy to
make in general. For example, every digit in
sage: N = 1000
sage: sin(RR(N))
0.826879540532003
is correct. However, for sufficiently large N, the results
sage: sum(sin(RR(n)) for n in range(0, N))
-0.0129099064588385
sage: sum(sin(RealField(100)(n)) for n in range(0, N))
-0.012909906458836372437031510981
are not correct in every digit. Any inexact numerical computation
should be taken with a grain of salt.
I think the doc test would be a *lot* more useful if there were
comments in the test like
# In Mathematica
# In[11]:= N[Integrate[ Sin[x]/x^2,{x,1,Pi/2}],50]
# Out[11]= 0.33944794097891567969192717186521861799447698826918
# This can also be computed in Wolfram Alpha, which uses Mathematica
# as a back end.
# http://www.wolframalpha.com/input/?i=N[Integrate[+Sin[x]%2Fx^2%2C{x
%2C1%2CPi%2F2}]%2C50]
# With mpmath
# sage: import mpmath
# sage: mpmath.mp.dps = 50
# sage: mpmath.quad(lambda x: mpmath.sin(x)/x**2, [1, mpmath.pi/2])
# mpf('0.33944794097891567969192717186521861799447698826917531')
# With Maple:
# some Result computed by Maple to arbitrary precision if
# someone knows how to do it.
Had this been done, then nobody in their right mind would have put
the expected result as 0.33944794097891573, since they would have
known those last two digits were inaccurate.
The answer "0.339447940978915..." succinctly lets the user know there
are some precision issues here.
I'm aware this might not be possible in all cases. If that is so,
then some explanation of why not, and why the result seems
'reasonable' would be appropriate. I'm not a mathematician, but I'm
well aware that for some calculations you can put an upper and lower
limit on a result given theories X and Y.
I for one would have increased confidence in Sage if I could see
that people had gone to the trouble of making independent checks of
the results in the documentation.
I agree here--the best doctests are those that have internal
consistency checks. This could involve trying out the numerical
routines against known closed-form solutions, verifying properties
such as linearity and additivity (especially if those are not obvious
consequences of the algorithms used), computing the same result to
higher precision, etc. In number theory, for example, there are often
strong consistency checks that are consequences of deep theorems that
can be demonstrated and verified (and that provides a more educational
docstring as well).
- Robert
--
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org
