Help with mode switching (create_pre_exit)

[Ilya Enkovich]

Hi,

I'm fighting with mode switching (to be more precise with
create_pre_exit function) trying to make it work for MPX. I saw
create_pre_exit had some stability issues  before and now I'm facing
similar issues trying to have it working when bound register is
returned by function in addition to GPR.

The more I look into create_pre_exit code, the more I think it makes
some assumptions which are wrong. Also it is very sensitive to the
code structure and simple changes in exit block crashes pre-exit block
creation. And in some cases it seems to me function does not fail by
accident.

Here is simple case on x86_64 with struct returned on registers:

struct S
{
  char *str;
  int n;
};

struct S foo()
{
  struct S s = {"", 0};
  return s;
}

Exit block here end with:

(insn 18 14 19 2 (set (reg:DI 0 ax)
        (reg:DI 80 [ <retval> ])) test.c:11 -1
     (nil))
(insn 19 18 22 2 (set (reg:SI 1 dx)
        (reg:SI 81 [ <retval>+8 ])) test.c:11 -1
     (nil))
(insn 22 19 23 2 (use (reg:DI 0 ax)) test.c:11 -1
     (nil))
(insn 23 22 0 2 (use (reg:SI 1 dx)) test.c:11 -1
     (nil))

create_pre_exit code assumes value is returned in a single register dx
which is wrong from the beginning. Then it does not realize set to dx
and ax are copy of returned value, finally reaches the end of BB and
splits there. It does not crash assuming it is a case with EH where
there are no return value copy in exit block.

If we exchange ax and dx usages then create_pre_exit will fail. If we
have value returned on more registers and thus will have more usages
then create_pre_exit will fail.

Now back to bound registers. In previous example exit block with MPX
would end like:

(insn 37 36 38 5 (set (reg:BND64 53 bnd0)
        (reg:BND64 85)) test.c:24 -1
     (nil))
(insn 46 42 47 5 (set (reg:DI 0 ax)
        (reg:DI 70 [ <retval> ])) test.c:25 -1
     (nil))
(insn 47 46 51 5 (set (reg:SI 1 dx)
        (reg:SI 71 [ <retval>+8 ])) test.c:25 -1
     (nil))
(insn 51 47 52 5 (use (reg:DI 0 ax)) test.c:25 -1
     (nil))
(insn 52 51 53 5 (use (reg:SI 1 dx)) test.c:25 -1
     (nil))
(insn 53 52 0 5 (use (reg:BND64 53 bnd0)) test.c:25 -1

There is an additional hard reg used for returned value and now
returned value is stored in non subsequent registers. I think that to
successfully cover all cases here, function_value target hook should
be used to determine returned hard regs, rather than use the last
'use' insn to determine required hard regs. Does it sound reasonable?
Am I missing something here?

Thanks,
Ilya