On 07/12/2010 02:52 AM, Robin Becker wrote:
A client wants to know why his db number -9.85 gets displayed by some
simple code as -9.8
I looked at the number and see that
>>> -9.85
-9.8499999999999996
ie I expect simple rounding to produce the observed result and indeed
>>> '%.1f' % -9.85
'-9.8'
however, when I use round I get an unexpected result ie
>>> round(-9.85,1)
-9.9000000000000004
according to its definition
round(x[, n])ΒΆ
Return the floating point value x rounded to n digits after the
decimal point.
If n is omitted, it defaults to zero. The result is a floating
point number.
Values are rounded to the closest multiple of 10 to the power
minus n;
if two multiples are equally close, rounding is done away from 0
(so. for example,
round(0.5) is 1.0 and round(-0.5) is -1.0).
so looking at the absolute differences I see
>>> abs(-9.9 - -9.85)
0.050000000000000711
>>> abs(-9.8 - -9.85)
0.049999999999998934
Do you *seriously* need to ask a computer if -9.85 is half-way between
-9.9 and -9.8. Just look at the numbers man! Of course it's equally
close to both, so rounding to -9.9 is correct according to the definition.
Do you actually *believe* that -9.9 - -9.85 = 0.050000000000000711.
Of course you know it's really 0.05. All you've done here is
demonstrate the fallible nature of (the last several digits of) floating
point arithmetic on a computer. Don't let that distract you from using
your common sense.
Gary Herron
ie the -9.8 value appears closer and at least to a primitive test
>>> abs(-9.9 - -9.85) > abs(-9.8 - -9.85)
True
the distance from the -9.9 result is larger, however, that may be
because the model numbers for -9.8 & -9.9 differ in distance from the
true 10**-n values eg
>>> -9.9
-9.9000000000000004
>>> -9.8
-9.8000000000000007
What value should round(-9.85,1) return? Is the result explainable in
python (ie without resort to the internal FP representations etc etc)?
--
http://mail.python.org/mailman/listinfo/python-list
