[Bug target/39240] New: Invalid sibcall optimization with promoted return types and differing signedness

[jakub at gcc dot gnu dot org]

extern void abort (void);

__attribute__ ((noinline))
static int foo1 (int x)
{
  return x;
}

__attribute__ ((noinline))
unsigned int bar1 (int x)
{
  return foo1 (x + 6);
}

unsigned long l1 = (unsigned int) -4;

__attribute__ ((noinline))
static short int foo2 (int x)
{
  return x;
}

__attribute__ ((noinline))
unsigned short int bar2 (int x)
{
  return foo2 (x + 6);
}

unsigned long l2 = (unsigned short int) -4;

__attribute__ ((noinline))
static signed char foo3 (int x)
{
  return x;
}

__attribute__ ((noinline))
unsigned char bar3 (int x)
{
  return foo3 (x + 6);
}

unsigned long l3 = (unsigned char) -4;

__attribute__ ((noinline))
static unsigned int foo4 (int x)
{
  return x;
}

__attribute__ ((noinline))
int bar4 (int x)
{
  return foo4 (x + 6);
}

unsigned long l4 = (int) -4;

__attribute__ ((noinline))
static unsigned short int foo5 (int x)
{
  return x;
}

__attribute__ ((noinline))
short int bar5 (int x)
{
  return foo5 (x + 6);
}

unsigned long l5 = (short int) -4;

__attribute__ ((noinline))
static unsigned char foo6 (int x)
{
  return x;
}

__attribute__ ((noinline))
signed char bar6 (int x)
{
  return foo6 (x + 6);
}

unsigned long l6 = (signed char) -4;

int
main (void)
{
  if (bar1 (-10) != l1)
    abort ();
  if (bar2 (-10) != l2)
    abort ();
  if (bar3 (-10) != l3)
    abort ();
  if (bar4 (-10) != l4)
    abort ();
  if (bar5 (-10) != l5)
    abort ();
  if (bar6 (-10) != l6)
    abort ();
  return 0;
}

aborts on powerpc64-linux, both -m32 and -m64, at -O2 and higher (whenever
-foptimize-sibling-calls is in effect).  In 4.4 this causes a Linux kernel
miscompilation on ppc64, see https://bugzilla.redhat.com/show_bug.cgi?id=485067
so to some extent it could be considered a regression, that said, the testcase
I'm providing here fails with 4.1, 4.3 as well as trunk.

As the psABI says that integral return values are passed in r3 and
zero-extended/sign-extended as needed to 64-bits (32-bits for 32-bit psABI),
obviously integral types with smaller precision than that need to have the same
signedness between caller and tail callee.

Just wonder whether this is something that should be fixed in rs6000
function_ok_for_sibcall hook (would probably be less risky for 4.4 and release
branches) or whether calls.c should do that generically.


-- 
           Summary: Invalid sibcall optimization with promoted return types
                    and differing signedness
           Product: gcc
           Version: 4.4.0
            Status: UNCONFIRMED
          Keywords: wrong-code
          Severity: normal
          Priority: P3
         Component: target
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: jakub at gcc dot gnu dot org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=39240