Steve Horsley wrote: > Ben O'Steen wrote: > > On Mon, October 31, 2005 10:23, Sybren Stuvel said: > >> Ben O'Steen enlightened us with: > >>> Using decimal as opposed to float sorts out this error as floats are > >>> not built to handle the size of number used here. > >> They can handle the size just fine. What they can't handle is 1/1000th > >> precision when using numbers in the order of 1e10. > > > > I used the word 'size' here incorrectly, I intended to mean 'length' > > rather than numerical value. Sorry for the confusion :) > > Sybren is right. The problem is not the length or the size, it's > the fact that 0.039 cannot be represented exactly in binary, in > just the same way that 1/3 cannot be represented exactly in > decimal. They both give recurring numbers. If you truncate those > recurring numbers to a finite number of digits, you lose > precision. And this shows up when you convert the inaccurate > number from binary into decimal representation where an exact > representation IS possible.
That's A source of error, but it's only part of the story. The double-precision binary representation of 0.039 is 5620492334958379 * 2**(-57), which is in error by 1/18014398509481984000. By contrast, Johnny Lee's answer is in error by 9/262144000, which is more than 618 billion times the error of simply representing 0.039 in floating point -- a loss of 39 bits. The problem here is catastrophic cancellation. 1130748744.500 ~= 4742703982051328 * 2**(-22) 1130748744.461 ~= 4742703981887750 * 2**(-22) Subtracting gives 163578 * 2**(-22), which has only 18 significant bits. -- http://mail.python.org/mailman/listinfo/python-list
