Philip Warner <[EMAIL PROTECTED]> writes:
> Now I understand, but it doesn't quite make sense given what was displayed.
> The float4 value is *displayed* as 10.1, not 10.101, so I had assumed
> that there was a level of either accuracy or display rouding happening.
In float4-to-ASCII, yes. M
At 16:12 7/08/00 +, Thomas Lockhart wrote:
>> I would have expected the latter to be at worst 10.10 +/-
>> .01.
>> Am I missing something?
>
>10.1 can't be represented exactly, so the float8 representation has bits
>set way down at the low end of the mantissa. When conv
> I would have expected the latter to be at worst 10.10 +/-
> .01.
> Am I missing something?
Well, yes :)
10.1 can't be represented exactly, so the float8 representation has bits
set way down at the low end of the mantissa. When converting to float4
those low bits get rou
