Issue 145801
Summary [Clang] Mixed use of VLAs and alloca leads to miscompilation (early pop of alloca memory)
Labels clang
Assignees
Reporter kkofler
    In this (minimized, not linkable, use `clang -O2 -S allocatest.c` and look at the output to test) testcase:
```c
#include <alloca.h>

struct nlopt;

extern int n;
struct nlopt *create(void);
void f(struct nlopt *opt, double *x, void *userdata);
void solve(struct nlopt *opt);

int main(void)
{
  struct nlopt *opt = create();
  int i;
  {
    double x[n];
    for (i = 0; i < n; i++) {
      void *p = alloca(8);
 f(opt, x, p);
    }
  }
  solve(opt);
  return 0;
}
```
the `alloca` needs to live until the end of the function, but (as evidenced by the assembly output), Clang actually emits code to restore `rsp` to its value before the VLA creation at the point where the VLA goes out of scope, and that also ends up destroying the arrays from `alloca`.

In the real (much longer) code, the `solve` function (actually called `nlopt_optimize`) ends up using the pointers that were recorded in the `opt` structure by the function `f` (actually called `nlopt_add_inequality_constraint`), but due to the early pop, they will have been overwritten, leading to a crash.

I can reproduce this both with the latest clang version 18.1.8 and with the ancient clang version 13.0.1, so this is a longstanding bug.

GCC gets this right: `gcc -O2 -S allocatest.c` shows that `rsp` is **not** restored until after `solve` is called (so the lifetime of the VLA also has to be extended, but there is not really a way around that in this case), and in fact, the real program also does not crash when compiled with GCC as it does when compiled with Clang.
_______________________________________________
llvm-bugs mailing list
llvm-bugs@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs

Reply via email to