davezarzycki added a comment. In D98179#2639476 <https://reviews.llvm.org/D98179#2639476>, @nlopes wrote:
> I'm talking about sorting just the summary of failed tests, not the whole > output. We need the whole -vv output, but that can be out of order. > > Why are timeouts important? Our use case is running Alive2 with the test > suite. Alive2 is heavy machinery and runs into timeouts. Running the tests in > roughly the same order every time is important to avoid timeout tests > flipping to failed or vice-versa. Plus slow tests = tests that consume a lot > of memory (in our scenario), so we can't bundle slow tests together. > Adding a `--ignore-timing-data` would be great, yes! But I still feel that > sorting the list of failed tests is important for user experience. I diff > these all the time. That still sounds incredibly brittle. If there is any variety in test machine performance, then any and all attempts at sorting should be futile because the underlying hardware will perturb different timeouts. Is this not your experience? How do you reconcile hardware performance and configuration details (like SMT) with timeout settings? Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D98179/new/ https://reviews.llvm.org/D98179 _______________________________________________ lldb-commits mailing list lldb-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
