| Issue |
145801
|
| Summary |
[Clang] Mixed use of VLAs and alloca leads to miscompilation (early pop of alloca memory)
|
| Labels |
clang
|
| Assignees |
|
| Reporter |
kkofler
|
In this (minimized, not linkable, use `clang -O2 -S allocatest.c` and look at the output to test) testcase:
```c
#include <alloca.h>
struct nlopt;
extern int n;
struct nlopt *create(void);
void f(struct nlopt *opt, double *x, void *userdata);
void solve(struct nlopt *opt);
int main(void)
{
struct nlopt *opt = create();
int i;
{
double x[n];
for (i = 0; i < n; i++) {
void *p = alloca(8);
f(opt, x, p);
}
}
solve(opt);
return 0;
}
```
the `alloca` needs to live until the end of the function, but (as evidenced by the assembly output), Clang actually emits code to restore `rsp` to its value before the VLA creation at the point where the VLA goes out of scope, and that also ends up destroying the arrays from `alloca`.
In the real (much longer) code, the `solve` function (actually called `nlopt_optimize`) ends up using the pointers that were recorded in the `opt` structure by the function `f` (actually called `nlopt_add_inequality_constraint`), but due to the early pop, they will have been overwritten, leading to a crash.
I can reproduce this both with the latest clang version 18.1.8 and with the ancient clang version 13.0.1, so this is a longstanding bug.
GCC gets this right: `gcc -O2 -S allocatest.c` shows that `rsp` is **not** restored until after `solve` is called (so the lifetime of the VLA also has to be extended, but there is not really a way around that in this case), and in fact, the real program also does not crash when compiled with GCC as it does when compiled with Clang.
![]()
_______________________________________________
llvm-bugs mailing list
llvm-bugs@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs
