Loren Wilton wrote: >> Erm.. Loren.. While that may be true of binary fractions, nobody uses >> binary fractions. >> >> In IEEE floating point format (single precision or otherwise), 0.001 has >> an exact binary representation. >> >> Very few things in this world use binary fractions. Standard floating >> point numbers on computers is one of them. >> > > Er, sorry, no. The value of .001 in IEEE double precision (64 bit > representation) is > 0x3F50624DD2F1A9FC. That most certainly has more than one bit set in the > mantissa. > > If I do the following: > > double val1 = .001; > float val2 = .001; > double val3 = val2 - val1; > > Then val3 has the value -4.7497451284573e-011, which is 0xbdca1cac08000000. > That looks to me suspiciously like a residue from a continuing fraction > represented in two different precisions.
Ok...Thinking some more here. The error from the above subtraction of mixed format is on the order of 5e-11. Technically the above number is not an exact capture of the error in a conversion from 0.001 to single-precision floating point, but the difference between the errors of conversion to single and double precision. However, the error in conversion to single is going to be massively larger than the error in conversion to double. Based on that, I'm assuming we can treat 5e-11 as the bound of worst-case error for conversion of 0.001 to floating point. However, SA rule scores can be up to 1000.0. This will cause a shift of the mantissa, and increase the magnitude of our error when adding 0.001 to 1000.0 to get 1000.001. Given that's an e6 shift, that brings our error up to 5e-5 per number added, more-or-less worst case. So, with single-precision floats, you could introduce an error of 0.001 by adding 20 SA scores together, more-or-less worst case. However... Perl uses double-precision floats for all its internal math, not single. Double-precision uses a 52-bit mantissa, compared to Single-precision with 23. This means the error introduced by double-precision floats is a factor of 2^29 smaller than single-precision floats. So in order to introduce a math error visible in rounding to 0.001 we'd need somewhere around 20*2^29 or about10 billion SA scores to be added together. I think it will take the SARE ninjas a long time to create enough rules that float conversions in perl will cause visible rounding errors, should SA ever convert to 3-decimal-place format. I made a LOT of logic jumps there, and it's all handled in a very rough-ballpark fashion but does anyone see any substantive errors in my thinking or math? (substantive here I mean enough that it would take less than 1 million scores added to cause a visible error? I'm sure my rough ballpark has significant error, but not by more than four orders of magnitude.)
