Endilll wrote: > Could you elaborate a bit more on that? What is the exact problem with the > testing infrastructure? Can we just add a separate test with the cases from > the issues, perhaps without -verify at all?
1) I think that the most reliable way to detect a crash would be to leverage our existing machinery for that inside Clang and its driver. Result this machinery produces is exit code. 2) We should keep in mind that on top of assertion failures, we also have segmentation faults. 3) Stack traces are also not a given, as can be seen in https://godbolt.org/z/cf7ovjrq3 (#31936) 4) Unfortunately, our exit code in case of crash is not consistent. On Linux, both driver and frontend return `143`. On Windows, frontend returns `-1073741819` (which is `0xc0000005`), which is converted to `1` by driver. The same `1` is used for regular failures due to errors. 5) From what I was able to find, `lit` can only check whether exit code is zero or non-zero (via `not`). `not` has `not --crash` mode that can also be leveraged. 6) This is not helped by the fact that by default piping (to e.g. `FileCheck`) ignores crashes. `lit` is following shell behavior here, but thankfully has `pipefail` option in `lit.cfg` that can help. While I think it's possible to craft a general solution out of what we have today, it seems too brittle and intricate. It's also possible to use a simpler solution that works only for e.g. assertion failures. I don't think either of the above are good options, especially when considered at scale. While it's a discussion for another day, we have hundreds (of not thousands) of regression test cases in our bug tracker labeled `crash`, `crash-on-valid`, and `crash-on-invalid`. Some of them have been long gone, but resurfaced during the last 2 years. So I'm not interested in a limited one-off solution just for this PR. Ideal solution seems to be along the following lines: 1) Make frontend consistently return e.g. `143` on all platforms in case of crash. 2) Make `lit` capable of checking specific exit code. https://github.com/llvm/llvm-project/pull/78898 _______________________________________________ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
