On 08/ 5/10 06:17 PM, Sergey Bochkanov wrote:
Hello,
Hmmm... Didn't thought about this situation yet. Definitely we can't
solve this problem with any kind of regular expressions. One
possible solution is to round data before printing. So both
1.00000000000001 and 0.99999999999999 will become 1.000000.
...however, we still can have problems when rounding X=0.000499999 up
to three digits. With original X we will have 0.000. But with
perturbation as small as 0.000000002 we will round to 0.001.
I've never done this, but the most logical thing to me seems to be to look at
the absolute magnitude of the relative error, for all cases except when the
expected value is 0. But I guess on cases where the numbers are not close to 1,
or other special cases one could think up, the current system is probably the
simplest.
Dave
--
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org
