[Bug tree-optimization/86853] New: sprintf optimization for wide strings doesn't account for conversion failure

[msebor at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86853

            Bug ID: 86853
           Summary: sprintf optimization for wide strings doesn't account
                    for conversion failure
           Product: gcc
           Version: 9.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: msebor at gcc dot gnu.org
  Target Milestone: ---

The handling of wide character and string constants in the sprintf pass fails
to consider the possibility of conversion failure.  As a result, it sets the
return value to a non-negative range when it should avoid setting it instead. 
(Pointed out in https://gcc.gnu.org/ml/gcc-patches/2018-08/msg00275.html).

$ cat d.c && gcc -O2 -Wall -fdump-tree-optimized=/dev/stdout d.c && ./a.out 
__WCHAR_TYPE__ ws[] = L"\uFFFF";

int main (void)
{
   int n0 = __builtin_snprintf (0, 0, "%S", L"\uFFFF");
   int n1 = __builtin_snprintf (0, 0, "%S", ws);

   __builtin_printf ("%i == %i\n", n0, n1);

   if (n0 != n1)
       __builtin_abort ();

}

;; Function main (main, funcdef_no=0, decl_uid=1899, cgraph_uid=1,
symbol_order=1) (executed once)

main ()
{
  int n1;

  <bb 2> [local count: 1073741825]:
  n1_5 = __builtin_snprintf (0B, 0, "%S", &ws);
  __builtin_printf ("%i == %i\n", 1, n1_5);
  if (n1_5 != 1)
    goto <bb 3>; [0.00%]
  else
    goto <bb 4>; [99.96%]

  <bb 3> [count: 0]:
  __builtin_abort ();

  <bb 4> [local count: 1073312327]:
  return 0;

}


1 == -1
Aborted (core dumped)