On 30/09/16 14:13 +0100, Szabolcs Nagy wrote:
On 30/09/16 10:35, Szabolcs Nagy wrote:
On 29/09/16 14:37, Andre Vieira (lists) wrote:
On arm-none-eabi I'm seeing a failure for the long double type and inputs:
{ 1e-2l, 1e-4l, 1e-4l, 0.010000999950004999375l }
The abs(frac) is higher than the toler: 1.73455e-16 vs 1e-16. Is that a
reasonable difference? Should we raise toler3 to 1e-15?
The last line is also too high:
{ 2147483647.l, 2147483647.l, 2147483647.l, 3719550785.027307813987l }
Yields a frac of: 1.28198e-16
Those are the only ones that pass the 1e-16 threshold.
i think the tolerance should be defined in
terms of LDBL_EPSILON (or numeric_limits).
n*LDBL_EPSILON tolerance would accept hypot
with about n ulp error.
now i see that there are huge ulp errors..
toler = 10*eps;
should work for all formats, but currently there
is >1000 ulp error on one of the double test cases..
so tolerance is carefully set to avoid triggering
the failure there
i'd set toler to 10000*eps if this test case is not
for testing hypot quality.
Ed is already re-working the code, and probably the tests.
I'd prefer to wait for his new implementation, but if the FAIL is a
problem now then go ahead and adjust the tolerance.
