On Fri, Dec 11, 2020 at 7:26 AM Xionghu Luo <luo...@linux.ibm.com> wrote:
>
> Thanks,
>
> On 2020/12/10 17:12, Richard Biener wrote:
> >> 2) From PR90070:
> >>
> >> double temp1 = (double)r->red;
> >> double temp2 = (double)aggregate.red;
> >> double temp3 = temp2 + (temp1 * 5.0);
> > temp1 * 5 could be not representable in float but the
> > result of the add could so the transform could result
> > in -+Inf where the original computation was fine (but
> > still very large result).
> >
> > Usually in such cases one could say we should implement some
> > diagnostic hints to the user that he might consider refactoring
> > his code to use float computations because we cannot really say
> > whether it's safe (we do at the moment not implement value-range
> > propagation for floating point types).
> >
>
> foo (double x, float y, float z)
> {
> return ( fabs (x) * y - z ) ;
> }
>
> int main ()
> {
> float res = foo (1e38, 5.0, 3e38);
> printf ("res:%f\n", res);
> }
>
> (1) $ gcc a.c -Ofast -ffp-contract=off:
>
> 0000000000000880 <foo>:
> 880: 10 0a 20 fc fabs f1,f1
> 884: b2 00 21 fc fmul f1,f1,f2
> 888: 28 18 21 fc fsub f1,f1,f3
> 88c: 18 08 20 fc frsp f1,f1
> 890: 20 00 80 4e blr
>
> $ ./a.out
> res:199999993605713849301312521538346418176.000000
>
> (2) $ gcc_MODIFIED a.c -Ofast -ffp-contract=off:
>
> 0000000010000660 <foo>:
> 10000660: 18 08 00 fc frsp f0,f1
> 10000664: 10 02 00 fc fabs f0,f0
> 10000668: b2 00 00 ec fmuls f0,f0,f2 // Inf
> 1000066c: 28 18 20 ec fsubs f1,f0,f3
> 10000670: 20 00 80 4e blr
>
> $ ./a.out
> res:inf
>
> It's true that if change all double computation to float will result
> in INF if "fabs (x) * y" is larger than FLT_MAX, though the double
> result in (1) could get back to a large number smaller than FLT_MAX.
>
>
> But the add/sub could also produces INF similarly,
>
> foo (double x, float y, float z)
> {
> return ( -fabs (x) + y + z ) ;
> }
>
> int main ()
> {
> float res = foo (1e38, 1e38, 3e38);
> printf ("res:%f\n", res);
> }
>
> (3) $ gcc a.c -Ofast:
>
> 0000000000000880 <foo>:
> 880: 10 0a 20 fc fabs f1,f1
> 884: 28 08 42 fc fsub f2,f2,f1
> 888: 2a 18 22 fc fadd f1,f2,f3
> 88c: 18 08 20 fc frsp f1,f1
> 890: 20 00 80 4e blr
>
> $ ./a.out
> res:300000000549775575777803994281145270272.000000
>
> 4) $ gcc_MODIFIED a.c -Ofast:
>
> 0000000010000660 <foo>:
> 10000660: 18 08 20 fc frsp f1,f1
> 10000664: 2a 18 42 ec fadds f2,f2,f3
> 10000668: 10 0a 20 fc fabs f1,f1
> 1000066c: 28 08 22 ec fsubs f1,f2,f1
> 10000670: 20 00 80 4e blr
>
> $ ./a.out
> res:inf
>
>
> Note that the add/sub sequence is different for (3) and (4) since
> -funsafe-math-optimizations is implicitly true. "fp-contract=fast" in
> (1) and (2) could avoid Inf as fmads could handle float overflow (verified
> it on Power, not sure other targets support this), but without float
> value-range info, it is unsafe to change computation from double to
> float even for only add/sub expressions.
Yes. As said it's difficult to second guess the programmer here.
The existing cases doing promotion look at unary functions,
doing exp->expf when arguments are promoted floats and the
return value is casted back to float. That's also a common programmer
"error" not knowing expf or assuming some kind of magic overloading.
So I'm not entirely convinced such transform is a good idea, at least
by default with -ffast-math. Maybe have a -fassume-float-limited-range
or so documented as that we assume that double or long double values
used fit in floats?
Richard.
> What's more, It seems also difficult to do computation *partly float
> and partly double* in backprop pass since all the expressions are
> chained and strong dependent unlike the sign-changing operations,
> which could only change expressions partly.
>
>
> Thanks,
> Xionghu
