On 29/09/16 14:37, Andre Vieira (lists) wrote:
>
> On arm-none-eabi I'm seeing a failure for the long double type and inputs:
> { 1e-2l, 1e-4l, 1e-4l, 0.010000999950004999375l }
>
> The abs(frac) is higher than the toler: 1.73455e-16 vs 1e-16. Is that a
> reasonable difference? Should we raise toler3 to 1e-15?
>
> The last line is also too high:
> { 2147483647.l, 2147483647.l, 2147483647.l, 3719550785.027307813987l }
> Yields a frac of: 1.28198e-16
>
> Those are the only ones that pass the 1e-16 threshold.
>
i think the tolerance should be defined in
terms of LDBL_EPSILON (or numeric_limits).
n*LDBL_EPSILON tolerance would accept hypot
with about n ulp error.
