Raúl Marín Rodríguez <rmrodriguez(at)carto(dot)com> wrote:
> I don't want to go too deep into it, but you get stuff like this:
>
> Select pow(2.0, -3)::text = pow(2, -3)::text;
> ?column?
> ----------
> f
Indeed, to me, that has turned out to be the most intriguing part of
the whole thread. Needs to be in some SQL subtleties exam somewhere:
select pow(2.0,-3), pow(2,-3);
pow | pow
--------------------+-------
0.1250000000000000 | 0.125
Looks like the first call resolves to the numeric version, while
the second (with integer arguments) resolves to the double one:
select pow(2.0,-3) is of (numeric), pow(2,-3) is of (double precision);
?column? | ?column?
----------+----------
t | t
Still, a numeric 0.125 doesn't always have those trailing zeros:
select pow(2.0,-3), pow(2,-3)::numeric;
pow | pow
--------------------+-------
0.1250000000000000 | 0.125
What's going on in the representation?
select numeric_send(pow(2.0,-3)), numeric_send(pow(2,-3)::numeric);
numeric_send | numeric_send
------------------------+------------------------
\x0001ffff0000001004e2 | \x0001ffff0000000304e2
I assume the 10 vs. 03 hex in the tails of those things represent
either 'precision' or 'scale' of 16 vs. 3? I don't get much help
from IS OF (numeric(p,s)), which seems to ignore any p,s and just
be true for any numeric. But here, this matches:
select numeric_send(0.125::numeric(16,16));
numeric_send
------------------------
\x0001ffff0000001004e2
How does numeric_power choose the precision and scale of its result?
Is that something the standard dictates?
Given that 0.125 is exact for this answer, at first I wanted to
ask if numeric_power could be made to produce the result with
precision 3, but then I realized that's backwards. A result with
precision 3 would be like saying, eh, it's somewhere between
0.1245 and 0.1255. If a result is known to be exact, it would be
better to go the other way and return it as numeric(huge).
That then led me to wonder if the cast float8_numeric is really
doing the right thing. Is it turning 0.125 (an exact representation
as float8) into numeric(3,3), again hedging as if it might be anything
from 0.1245 to 0.1255? Would it be better for float8_numeric to
produce a numeric with the precision/scale reflecting the actual
limits of float8?
Ok, now I've been driven to UTSL. It looks as if the intent of
the snprintf(..., "%.*g", DBL_DIG, val) in float8_numeric could
have been to accomplish that. It doesn't, though, as (at least
on my platform), %g drops trailing zeros, though there
is a documented 'alternate form' flag # that prevents that.
It works in bash:
bash-4.2$ printf '%.*g\n' 15 0.125
0.125
bash-4.2$ printf '%#.*g\n' 15 0.125
0.125000000000000
Does the standard prescribe how cast(float8 as numeric) ought to
select the precision/scale?
Sorry to drift OT, as this is more about the SQL functions than
pgbench, but it was too puzzling to ignore. :)
-Chap
