Gregory Stark <[EMAIL PROTECTED]> writes:
> In the case reported div_var was getting 70/70 = 0.99999. Which is
> really just wrong.
Agreed, but I think your proposed patch is just a band-aid. The real
problem with div_var is that it generates inaccurate output digits at
all --- it's assuming that computing a few guard digits before rounding
will suffice to ensure that all the delivered digits are correct, but
I think there will always be corner cases where it fails.
I just blew the dust off my old copy of Knuth vol 2, and see that his
algorithm for multi-precision division generates output digits that are
correct to start with (or at least he never needs to revisit a digit
after moving on to the next). ISTM we should go over to an approach
like that. For the truncated-output case the result will be exact,
and for the rounded-output case you generate exactly one guard digit
to see if it's >= base/2.
regards, tom lane
---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
