On Friday 26 January 2001 18:07, Tom Lane wrote:
> Curiously, this change exposed what I take to be a platform dependency
> in the int8 regress test. It was computing
> int8(float8(4567890123456789::int8)) and expecting to get back exactly
> 4567890123456789. However, that value is 53 bits long
>> The fact that 5*27.81*100 != 27.81*100*5 is certainly a garden-variety
>> floating-point roundoff error. However, I think Max has a fair
>> complaint here: it seems float-to-int8 conversion is truncating, not
>> rounding like the other conversions to integer do.
I have changed float8-to-int8
Notice how the INT4 rounding is banker's rounding (round to the nearest even
number). That is what we would want the INT8 to do as well, not just a
simple round like I mentioned before. Again, the INT8 shows truncation. I've
been looking around the source code, but I can't see where all this
On Thursday 25 January 2001 22:52, Tom Lane wrote:
> "Robert B. Easter" <[EMAIL PROTECTED]> writes:
> > This problem is not specific to Postgres.
>
> The fact that 5*27.81*100 != 27.81*100*5 is certainly a garden-variety
> floating-point roundoff error. However, I think Max has a fair
> complaint
"Robert B. Easter" <[EMAIL PROTECTED]> writes:
> This problem is not specific to Postgres.
The fact that 5*27.81*100 != 27.81*100*5 is certainly a garden-variety
floating-point roundoff error. However, I think Max has a fair
complaint here: it seems float-to-int8 conversion is truncating, not
ro
This problem is not specific to Postgres. If you play around with a little C
program like:
#include
int main(int argc, char * argv[])
{
float f = 27.81;
int i = 5;
int l = 100;
int ii = i*f*l;
long ll = l*f*i;
float ff = i*f*l;
print
