Bill, Thanks for getting to the bottom of this. It's about time. People have asked about this probably 10 times before...
-- William On Fri, 27 Oct 2006 07:57:10 -0500, Bill Hart <[EMAIL PROTECTED]> wrote: > > > William Stein wrote: > >> Literals are parsed by the RealNumber command, which you can set to >> whatever >> you want (as I explained a day or two ago on sage-devel). > > Whoops, I see that now. I can't imagine how I managed to miss that. It > works, thanks. > > By the way, I noted above that one more decimal place was being > displayed for floats of the "default" precision of 53 bits than should > be. This was slightly incorrect. > > This was also based on the assumption that the sign is not stored in > that 53 bits, but I think it actually is stored there. Thus, here is my > revised analysis: > > The binary value for a precision of 53 bits including a sign is correct > to one part in 2^52. We want that to be less than the smallest decimal > to be displayed. Now 1/2^52 = 2.22E-16. Thus for a decimal mantissa of > 9.9999999... one wants the last digit displayed to be bigger than > 10*2.22E^-16 = 2.22E-15, i.e. the last decimal digit of the mantissa > displayed should be 1E-14, which implies there should be 15 significant > decimal digits displayed. There are currently 17 displayed. Thus this > needs to be reduced by 2. > > This example should give a suitable formula for any binary precision. > > If I am wrong about the sign being stored in the mantissa then this > will obviously change the analysis (though I think the result is the > same for a 53 bit number, though very few would actually need two less > decimal digits and most would be displayed correctly with just one > less). > > The default precision should be 53 bits, since that is the size of the > mantissa that can be stored in a 64 bit machine word, and so will be > optimal from a speed point of view on 32 and 64 bit architectures > (which most are). > > I don't know what formula is currently being used to determine the > number of digits being displayed, so I can't say how it should be > altered, but I'll leave that to those familiar with that part of the > SAGE code. > > Also above someone noted that the documentation says that the default > is for rounding down but the reality is that it is rounded up (until > William changed one or the other). However Knuth recommends that to > prevent "floating point drift", the default should be to round down > rather than up. It wasn't clear to me whether this meant round all > floats down or only round those down that are smaller than *or equal* > to the 0.5 for the next decimal place. Anyhow, the MPFR documentation > explains this and what the various rounding modes do for you. At any > rate, it is possible that the SAGE documentation regarding the default > rounding mode needs to be changed, not the actual code. > > Hope the above is helpful. > > Bill. > > > > --~--~---------~--~----~------------~-------~--~----~ To post to this group, send email to sage-devel@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sage-devel URLs: http://sage.scipy.org/sage/ and http://modular.math.washington.edu/sage/ -~----------~----~----~----~------~----~------~--~---
