On Thu, 30 Oct 2014, Frédéric Parrenin wrote:
OK I see.
But why this problem does not show up in Calc or XL?
It seems they manage to find a workaround.
They're applying some cosmetic heuristic. If you were to type
0.6000000000000001 into a cell, it would show as 0.6000000000000000.
I don't know if these programs have an option "Show the number
actually stored" but they ought to!
Allin Cottrell
2014-10-30 18:13 GMT+01:00 Allin Cottrell <[email protected]>:
On Thu, 30 Oct 2014, Frédéric Parrenin wrote:
If I type 0 in A1 and 0.1 in A2, select A1:A2 and then drag the bottom
right corner, I get:
0
0.1
0.2
0.3
0.4
0.5
0.6000000000000001
I am using 1.12.14 on a linux 64 bit machine.
Is this a known issue?
If yes, what is the corresponding bug report?
Yes, it's a very well known issue, but there's no corresponding bug report
-- because the issue is the inherently finite precision of computer
arithmetic. The "double" datatype comprises 64 bits and cannot model the
real line. You've just discovered that 0.6000... (with an arbitrary number
of trailing zeros) is not a number that can be exactly represented by a
double.
Allin Cottrell
--
Allin Cottrell
Department of Economics
Wake Forest University, NC
_______________________________________________
gnumeric-list mailing list
[email protected]
https://mail.gnome.org/mailman/listinfo/gnumeric-list
