Hi,
i386elf.h defines:
/* The ELF ABI for the i386 says that records and unions are returned
in memory. */
#define SUBTARGET_RETURN_IN_MEMORY(TYPE, FNTYPE) \
(TYPE_MODE (TYPE) == BLKmode \
|| (VECTOR_MODE_P (TYPE_MODE (TYPE)) && int_size_in_bytes (TYPE) == 8))
and as such differs from the regular i86 return mechanism. Notice that the
comment doesn't match the code:
*) some structs/unions are non BLKmode
*) some vectors can be BLKmode, some might not be -- the vector mode check
appears to be an attempt to catch DImode vectors.
Basing your ABI on the internal modes used by the compiler is not, IMHO, a
sensible design choice.
This code doesn't appear at first glance to cope with transparent_union. In
fact it looks pretty bitrotted.
Is it best just to junk the different behaviour at this point?
nathan